Segment Status - Incident History

<?xml version="1.0" encoding="UTF-8"?> <feed xml:lang="en-US" xmlns="http://www.w3.org/2005/Atom"> <id>tag:status.segment.com,2005:/history</id> <link rel="alternate" type="text/html" href="https://status.segment.com"/> <link rel="self" type="application/atom+xml" href="https://status.segment.com/history.atom"/> <title>Segment Status - Incident History</title> <updated>2025-02-22T12:32:42-08:00</updated> <author> <name>Segment</name> </author> <entry> <id>tag:status.segment.com,2005:Incident/24073055</id> <published>2025-02-20T10:00:00-08:00</published> <updated>2025-02-21T12:03:08-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/7dfbjfqg39y1"/> <title>Reverse ETL Database Upgrade</title> <content type="html">THIS IS A SCHEDULED EVENT Feb <var data-var='date'>20</var>, <var data-var='time'>10:00</var> - <var data-var='time'>12:00</var> PSTFeb <var data-var='date'>20</var>, <var data-var='time'>13:11</var> PST Update - Reverse ETL DB upgrade for EU is now complete. All systems are healthy. Next, we will conduct US upgrade on Feb 26, at 12 AM GMTFeb <var data-var='date'>19</var>, <var data-var='time'>10:33</var> PST Update - We will be undergoing scheduled maintenance during this time.Feb <var data-var='date'>19</var>, <var data-var='time'>10:28</var> PST Scheduled - We will be conducting a scheduled database upgrade. The maintenance window is expected to last approximately 2 hours. • EU: Starting at 06:00 PM GMT on Thursday, Feb 20, 2025 • US: Starting at 12:00 AM GMT on Wednesday, Feb 26, 2025 During this time there will be impact to the Reverse ETL, Linked Audiences, and Linked Events products. Reverse ETL • Syncs scheduled during the maintenance will be delayed. • RETL syncs initiated manually will not be processed. Linked Audiences • Syncs scheduled during the maintenance will be delayed. Linked Events • Syncs scheduled during the maintenance will be delayed. No other part of Segment will be impacted.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23903642</id> <published>2025-02-13T00:00:57-08:00</published> <updated>2025-02-13T12:34:47-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/zwjx295myx3t"/> <title>Delivery Failure Samples are Unavailable</title> <content type="html">Feb <var data-var='date'>13</var>, <var data-var='time'>00:00</var> PST Resolved - This incident has been resolved.Feb <var data-var='date'>10</var>, <var data-var='time'>12:00</var> PST Investigating - As we investigate a technical issue, Delivery Samples in Event Delivery, Delivery Overview and in Failed RETL syncs are temporarily unavailable.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23864915</id> <published>2025-02-05T23:47:25-08:00</published> <updated>2025-02-05T23:47:25-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/0btxlf4ly5m2"/> <title>Error Samples in Event Delivery and Delivery Overview are temporarily unavailable</title> <content type="html">Feb <var data-var='date'> 5</var>, <var data-var='time'>23:47</var> PST Resolved - This incident has been resolved.Feb <var data-var='date'> 5</var>, <var data-var='time'>21:35</var> PST Identified - As we investigate an issue, Delivery Samples in Event Delivery and Delivery Overview are unavailable.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23847395</id> <published>2025-02-04T14:38:32-08:00</published> <updated>2025-02-04T14:38:32-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/kbdkz7nc3ht9"/> <title>Warehouse Destination Delay</title> <content type="html">Feb <var data-var='date'> 4</var>, <var data-var='time'>14:38</var> PST Resolved - This incident has been resolved.Feb <var data-var='date'> 4</var>, <var data-var='time'>12:16</var> PST Monitoring - We have identified an issue with Warehouse Destinations that affects US workspace customers. Data delivery is delayed. A fix has been implemented and we are monitoring the result.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23684927</id> <published>2025-01-24T16:54:56-08:00</published> <updated>2025-01-24T16:54:56-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/ltpywsxs0x17"/> <title>Delay in Profiles Identity Resolution</title> <content type="html">Jan <var data-var='date'>24</var>, <var data-var='time'>16:54</var> PST Resolved - The delay in the identity resolution pipeline has been resolved.Jan <var data-var='date'>24</var>, <var data-var='time'>13:08</var> PST Monitoring - Fix is deployed. We are seeing improvements and will continue to monitor the recovery.Jan <var data-var='date'>24</var>, <var data-var='time'>11:16</var> PST Identified - We have identified the case for the issue and are working on a fix.Jan <var data-var='date'>24</var>, <var data-var='time'>10:57</var> PST Investigating - We have detected in an increase in delay in identity resolution and are actively investigating.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23596790</id> <published>2025-01-17T14:47:54-08:00</published> <updated>2025-01-17T14:51:00-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/3t83f9fxx372"/> <title>Processing delays for real-time Audiences, Computed Traits, and Journeys</title> <content type="html">Jan <var data-var='date'>17</var>, <var data-var='time'>14:47</var> PST Resolved - This incident is now resolved, and Audiences, Journeys and Computed Traits are processing data as expected.Jan <var data-var='date'>17</var>, <var data-var='time'>14:23</var> PST Update - We are continuing to monitor the recovery process and estimate that we will be fully back to real-time within the next 30 minutes.Jan <var data-var='date'>17</var>, <var data-var='time'>11:18</var> PST Update - We are continuing to monitor the recovery process and estimate that we will be fully back to real-time within 4 hours.Jan <var data-var='date'>17</var>, <var data-var='time'>10:28</var> PST Monitoring - We're currently experiencing delays of up to 7 hours processing messages for some Engage Audiences, Computed Traits, and Journeys. The issue has been identified and the recovery process has started. No data has been lost. Some audience and journey conditions using time windows of 7 hours or less may not have output values during this incident.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23164668</id> <published>2024-12-11T20:32:13-08:00</published> <updated>2024-12-11T20:32:13-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/l65y6xh97znd"/> <title>Generative AI features unavailable</title> <content type="html">Dec <var data-var='date'>11</var>, <var data-var='time'>20:32</var> PST Resolved - This incident has been resolved.Dec <var data-var='date'>11</var>, <var data-var='time'>16:21</var> PST Identified - Generative Audiences, Suggested Mappings, and Functions Writer and Analyzer are currently unavailable due to an AI vendor outage.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23081824</id> <published>2024-12-09T06:43:06-08:00</published> <updated>2024-12-09T06:43:06-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/2np0qzxf0ss1"/> <title>Engage batch computations, Predictions and Recommendations may produce incomplete results.</title> <content type="html">Dec <var data-var='date'> 9</var>, <var data-var='time'>06:43</var> PST Resolved - The missing links have been fully recovered. Batch computations and Predictions will automatically include the updates in the subsequent runs. There has been no impact to the Profile API and real-time computations. We apologize for the inconvenience.Dec <var data-var='date'> 6</var>, <var data-var='time'>06:52</var> PST Monitoring - We identified and resolved an issue where a small percentage of profiles lost links post-merge, affecting event attribution to canonical profiles. There is no data loss. We are working to recover the missing links for accurate profile and computations updates.Dec <var data-var='date'> 5</var>, <var data-var='time'>09:52</var> PST Identified - The issue has been identified.Dec <var data-var='date'> 5</var>, <var data-var='time'>08:52</var> PST Update - We are continuing to investigate this issue.Dec <var data-var='date'> 5</var>, <var data-var='time'>07:58</var> PST Investigating - We are currently investigating this issue.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/23082499</id> <published>2024-12-05T14:38:33-08:00</published> <updated>2024-12-05T14:38:33-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/f8602qmj92mq"/> <title>Some Unify and Engage customers are experiencing inability to create new Audiences and Traits</title> <content type="html">Dec <var data-var='date'> 5</var>, <var data-var='time'>14:38</var> PST Resolved - This incident has been resolvedDec <var data-var='date'> 5</var>, <var data-var='time'>08:59</var> PST Investigating - We are currently investigating this issue.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22851297</id> <published>2024-11-13T05:24:56-08:00</published> <updated>2024-11-13T08:26:32-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/4646kv46zr6p"/> <title>Identifier Traits not populating as Identifiers for Custom ID Rules</title> <content type="html">Nov <var data-var='date'>13</var>, <var data-var='time'>05:24</var> PST Resolved - On Nov 11, from 6:38AM UTC to Nov 13 12:45PM UTC, customers may have experienced an issue where their users were unable to get custom identifiers added to their profiles. Our engineers have identified and deployed a fix. We apologize for any inconvenience this may have caused.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22801930</id> <published>2024-11-08T05:45:10-08:00</published> <updated>2024-11-08T05:45:11-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/d2vjb459jfdv"/> <title>Sync Failures of AWS destinations</title> <content type="html">Nov <var data-var='date'> 8</var>, <var data-var='time'>05:45</var> PST Resolved - The issue has now been resolved and Syncs to amazon-kinesis-firehose, amazon-personalize, amazon-lambda and amazon-eventbridge are now flowing.Nov <var data-var='date'> 8</var>, <var data-var='time'>03:19</var> PST Update - We have also now recovered AWS Lambda. We will continue working through and monitoring the issue.Nov <var data-var='date'> 8</var>, <var data-var='time'>03:01</var> PST Monitoring - amazon-kinesis-firehose and amazon-eventbridge have recovered - we will continue monitoring while working to recover amazon-lambdaNov <var data-var='date'> 8</var>, <var data-var='time'>03:00</var> PST Identified - We have identified an issue with Syncs to amazon-kinesis-firehose, amazon-personalize, amazon-lambda and amazon-eventbridge and working to resolve.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22796358</id> <published>2024-11-07T14:42:49-08:00</published> <updated>2024-11-07T14:42:50-08:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/bvr140kb658l"/> <title>Processing delays for real-time Audiences, Computed Traits, and Journeys</title> <content type="html">Nov <var data-var='date'> 7</var>, <var data-var='time'>14:42</var> PST Resolved - We have successfully mitigated the issue and are fully back to real-time processing.Nov <var data-var='date'> 7</var>, <var data-var='time'>13:47</var> PST Update - We are continuing to monitor the recovery process and estimate that we will be fully back to real-time within 1 hour.Nov <var data-var='date'> 7</var>, <var data-var='time'>13:23</var> PST Monitoring - We're currently experiencing delays of up to 4 hours processing messages for 53.67% of Engage Audiences, Computed Traits, and Journeys. The issue has been identified and the recovery process has started. We estimate 38% of the affected computations will be back to real-time in the next 30 minutes, and the remainder will be back to real-time within 3 hours. No data has been lost.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22727572</id> <published>2024-11-02T13:13:34-07:00</published> <updated>2024-11-02T13:13:34-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/38sch9vt1rmg"/> <title>Scheduled maintenance on Nov 2nd from 10 AM PST - 1 PM PST</title> <content type="html">Nov <var data-var='date'> 2</var>, <var data-var='time'>13:13</var> PDT Resolved - Maintenance complete.Oct <var data-var='date'>31</var>, <var data-var='time'>12:32</var> PDT Monitoring - We will be conducting scheduled maintenance on our core configuration management database. During this period, the App and Public API will be read-only and login functionality will not be working.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22726359</id> <published>2024-10-31T09:30:00-07:00</published> <updated>2024-10-31T10:19:33-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/jfczrp0ntcn4"/> <title>EU - Event Delivery Delayed</title> <content type="html">Oct <var data-var='date'>31</var>, <var data-var='time'>09:30</var> PDT Resolved - We identified and resolved an isolated event delivery delay in our EU environment that caused delays for 10% of connections traffic up to 40 minutes.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22668280</id> <published>2024-10-25T15:00:19-07:00</published> <updated>2024-10-25T15:00:20-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/8hqlqgq4v8r9"/> <title>Syncs to Data Lakes are taking longer than expected.</title> <content type="html">Oct <var data-var='date'>25</var>, <var data-var='time'>15:00</var> PDT Resolved - This incident has been resolved.Oct <var data-var='date'>25</var>, <var data-var='time'>12:02</var> PDT Monitoring - A fix has been implemented and we are monitoring the result.Oct <var data-var='date'>25</var>, <var data-var='time'>09:10</var> PDT Identified - We have identified an issue with Data Lakes that affects some customers. Syncs to Data Lakes are taking longer than expected. No data loss has occurred.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22646004</id> <published>2024-10-23T14:59:51-07:00</published> <updated>2024-10-23T14:59:52-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/xxhy0h7bb5k7"/> <title>Engage Batch Computations High Error Rate</title> <content type="html">Oct <var data-var='date'>23</var>, <var data-var='time'>14:59</var> PDT Resolved - This incident has been resolved.Oct <var data-var='date'>23</var>, <var data-var='time'>11:31</var> PDT Monitoring - A fix has been implemented and we are monitoring the results.Oct <var data-var='date'>23</var>, <var data-var='time'>10:34</var> PDT Investigating - We are currently experiencing a high error rate with Engage batch computations.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22382900</id> <published>2024-10-04T14:09:01-07:00</published> <updated>2024-10-04T14:09:01-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/40f10j7wh9z6"/> <title>Increased Engage List destinations error rate</title> <content type="html">Oct <var data-var='date'> 4</var>, <var data-var='time'>14:09</var> PDT Resolved - This incident has been resolved.Oct <var data-var='date'> 3</var>, <var data-var='time'>15:47</var> PDT Monitoring - A fix has been implemented and we are monitoring results.Oct <var data-var='date'> 3</var>, <var data-var='time'>14:27</var> PDT Identified - The issue has been identified and a fix is being implemented.Oct <var data-var='date'> 3</var>, <var data-var='time'>13:36</var> PDT Investigating - We are currently investigating this issue.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22385552</id> <published>2024-10-04T13:50:44-07:00</published> <updated>2024-10-04T13:50:45-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/9dlr9z7kvkt7"/> <title>Delay in sending events to third party destinations</title> <content type="html">Oct <var data-var='date'> 4</var>, <var data-var='time'>13:50</var> PDT Resolved - This incident has been resolved.Oct <var data-var='date'> 3</var>, <var data-var='time'>19:31</var> PDT Monitoring - A fix has been implemented and we are monitoring the results.Oct <var data-var='date'> 3</var>, <var data-var='time'>19:11</var> PDT Identified - The issue has been identified and a fix is being implemented.Oct <var data-var='date'> 3</var>, <var data-var='time'>19:02</var> PDT Investigating - We are currently investigating the issue.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22157172</id> <published>2024-09-13T09:28:20-07:00</published> <updated>2024-09-13T09:28:21-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/5h4g52ns06p3"/> <title>Increased error rates of Profiles API in EU</title> <content type="html">Sep <var data-var='date'>13</var>, <var data-var='time'>09:28</var> PDT Resolved - This incident has been resolved.Sep <var data-var='date'>13</var>, <var data-var='time'>08:05</var> PDT Identified - We have identified an issue and deployed a fix. Profiles API in EU is recovering.Sep <var data-var='date'>13</var>, <var data-var='time'>07:12</var> PDT Investigating - We detected increased error rate in Profile API for our EU region service and are actively investigating.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/22127824</id> <published>2024-09-10T19:50:34-07:00</published> <updated>2024-09-10T19:50:34-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/v83qymcn9vy2"/> <title>Data processing delay in Twilio Engage</title> <content type="html">Sep <var data-var='date'>10</var>, <var data-var='time'>19:50</var> PDT Resolved - This incident has been resolved.Sep <var data-var='date'>10</var>, <var data-var='time'>18:49</var> PDT Monitoring - We detected a an issue that's causing a delay in data processing in Engage. A fix has been deployed and we are monitoring closely.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/21959827</id> <published>2024-08-29T02:00:00-07:00</published> <updated>2024-08-29T03:42:51-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/rc7lzrl3yb3w"/> <title>Amazon Kinesis Stream delays</title> <content type="html">Aug <var data-var='date'>29</var>, <var data-var='time'>02:00</var> PDT Resolved - Delivery delays to Amazon Kinesis of over an hour due to AWS outage.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/21759649</id> <published>2024-08-15T20:22:30-07:00</published> <updated>2024-08-15T20:22:31-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/g3kzctdm95mv"/> <title>Databricks Reverse ETL and Destination Sync delays</title> <content type="html">Aug <var data-var='date'>15</var>, <var data-var='time'>20:22</var> PDT Resolved - This incident has been resolved.Aug <var data-var='date'>15</var>, <var data-var='time'>20:03</var> PDT Monitoring - A fix has been implemented and we are monitoring the results.Aug <var data-var='date'>15</var>, <var data-var='time'>19:18</var> PDT Investigating - We are investigating an issue with Databricks ReverseETL and Destination that is affecting all customers.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/21620945</id> <published>2024-08-03T03:46:54-07:00</published> <updated>2024-08-03T03:46:54-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/q4ylgrvw69q2"/> <title>Incident impacting EU</title> <content type="html">Aug <var data-var='date'> 3</var>, <var data-var='time'>03:46</var> PDT Resolved - All systems have recovered. We have resolved this incidentAug <var data-var='date'> 3</var>, <var data-var='time'>03:41</var> PDT Monitoring - We identified the root cause and fixed the issue. Our systems are recovering. We continue to monitor the recoveryAug <var data-var='date'> 3</var>, <var data-var='time'>01:34</var> PDT Investigating - We have an incident impacting many products in the EU. We are investigating this. Multiple products such as Engage, rETL, Profile Sync are impacted in the EU. At this time the scope of impact is a delay in the impacted services.</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/21576874</id> <published>2024-07-31T09:04:48-07:00</published> <updated>2024-07-31T09:04:49-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/vnx7gbrckyt2"/> <title>Heap.io Partner Outage</title> <content type="html">Jul <var data-var='date'>31</var>, <var data-var='time'>09:04</var> PDT Resolved - Heap.io confirmed that the AWS outage affecting data delivery operation is now resolved.Jul <var data-var='date'>30</var>, <var data-var='time'>16:28</var> PDT Investigating - Our partner Heap.io is having an outage. We are monitoring the situation. For heap specific information visit https://www.heap.io/status</content> </entry> <entry> <id>tag:status.segment.com,2005:Incident/21560098</id> <published>2024-07-29T09:27:58-07:00</published> <updated>2024-07-29T09:27:58-07:00</updated> <link rel="alternate" type="text/html" href="https://status.segment.com/incidents/hhtgbzyjmqz7"/> <title>Engage Premier Email Template Creation and Editing Issue</title> <content type="html">Jul <var data-var='date'>29</var>, <var data-var='time'>09:27</var> PDT Resolved - A fix has been implemented, and we are continuing to monitor the result.Jul <var data-var='date'>29</var>, <var data-var='time'>05:38</var> PDT Investigating - The Segment engineering team has acknowledged an incident affecting customers with Engage Premier. These customers are currently unable to create new email templates or edit existing ones. When attempting to do so, an error message stating "Something went wrong" appears. We are actively working on resolving this issue and will provide updates as soon as possible. Thank you for your patience.</content> </entry> </feed>

CINXE.COM

Segment Status - Incident History