tag:status.flightcontrol.dev,2005:/historyFlightcontrol Status - Incident History2024-03-28T11:47:55-04:00Flightcontroltag:status.flightcontrol.dev,2005:Incident/203667392024-03-26T19:20:05-04:002024-03-26T19:20:05-04:00Deployments failing<p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>19:20</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>19:00</var> EDT</small><br><strong>Update</strong> - The fix is implemented and deploying</p><p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>18:13</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.flightcontrol.dev,2005:Incident/197596772024-01-18T20:29:13-05:002024-01-18T20:29:13-05:00Dashboard inaccessible<p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>20:29</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>20:24</var> EST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>20:13</var> EST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.flightcontrol.dev,2005:Incident/196701532024-01-09T19:30:30-05:002024-01-10T13:28:40-05:00Delay in processing deployments<p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>19:30</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>19:00</var> EST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>19:00</var> EST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>18:00</var> EST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>17:30</var> EST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.flightcontrol.dev,2005:Incident/190125282023-11-03T13:34:46-04:002023-11-03T13:34:46-04:00New Dashboard showing an error instead of projects<p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>13:34</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>13:23</var> EDT</small><br><strong>Identified</strong> - We've identified the problem with the new dashboard. In the meantime, the old dashboard at https://legacy.flightcontrol.dev is fully operational.</p>tag:status.flightcontrol.dev,2005:Incident/188322992023-10-17T18:10:53-04:002023-10-17T18:30:31-04:00Flightcontrol Deployments Not Completing<p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>18:10</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>17:40</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.flightcontrol.dev,2005:Incident/177187682023-06-29T08:32:33-04:002023-06-29T08:32:33-04:00Preview Deployments Failing With Environment Not Found<p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>08:32</var> EDT</small><br><strong>Resolved</strong> - The incident has now been resolved. To fix previous deployment errors in a preview environment, you can either trigger a manual deployment from the Flightcontrol dashboard or push a new commit to your branch.</p><p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>07:17</var> EDT</small><br><strong>Monitoring</strong> - We have deployed the fix and we are monitoring. New commits pushed to your preview environments should correctly trigger a deployment.</p><p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>06:52</var> EDT</small><br><strong>Update</strong> - The fix has been implemented and we are in the process of testing it.</p><p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>06:36</var> EDT</small><br><strong>Identified</strong> - We have identified an issue that affects preview deployments and we are in the process of fixing the issue.</p>tag:status.flightcontrol.dev,2005:Incident/176235962023-06-19T17:36:00-04:002023-06-19T17:36:00-04:00Partial Outage with Flightcontrol Deployments<p><small>Jun <var data-var='date'>19</var>, <var data-var='time'>17:36</var> EDT</small><br><strong>Resolved</strong> - This incident related to builds and environment variables has been resolved, and builds should now be performing successfully.</p><p><small>Jun <var data-var='date'>19</var>, <var data-var='time'>17:23</var> EDT</small><br><strong>Monitoring</strong> - Our engineering team has implemented a fix to this build/environment variables issue, and we are now watching deployment activity closely for errors. We suggest you rerun any failed deployments due to this issue.</p><p><small>Jun <var data-var='date'>19</var>, <var data-var='time'>17:05</var> EDT</small><br><strong>Update</strong> - We've prepared a fix for the issue, and are in the process of testing the fix. Any build errors related to missing environment variables are likely due to this issue. After our fix is deployed, any failed builds would need to be run again.</p><p><small>Jun <var data-var='date'>19</var>, <var data-var='time'>16:40</var> EDT</small><br><strong>Identified</strong> - We have identified an issue that affects some Flightcontrol deployments during the build process, and we are in the process of fixing the issue.</p>tag:status.flightcontrol.dev,2005:Incident/174846932023-06-05T22:16:39-04:002023-06-05T22:17:36-04:00Deployment Fail with Git clone error<p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>22:16</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved. Issue confirmed to be related to network issue on AWS side.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>22:05</var> EDT</small><br><strong>Monitoring</strong> - Issue seems to be rectified, and systems are back to operational. We will continue to monitor our systems. Issue is related to multiple network errors.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>21:51</var> EDT</small><br><strong>Update</strong> - Issue seems to be related to connectivity on AWS us-west-2</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>21:32</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>21:32</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating reports with deployment failure due to Git clone error</p>tag:status.flightcontrol.dev,2005:Incident/167601812023-04-05T10:45:11-04:002023-04-05T12:10:49-04:00Manual Deployments are stuck at specific commit<p><small>Apr <var data-var='date'> 5</var>, <var data-var='time'>10:45</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'> 5</var>, <var data-var='time'>10:40</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Apr <var data-var='date'> 5</var>, <var data-var='time'>10:29</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.flightcontrol.dev,2005:Incident/167533882023-04-04T17:28:09-04:002023-04-04T17:28:09-04:00Issue with Builds<p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>17:28</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>17:20</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>16:58</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>16:58</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>16:55</var> EDT</small><br><strong>Investigating</strong> - We are fixing an issue related to Build. Build fails with an error message similar to the below:<br />Error: ENOENT: no such file or directory, open '/tmp/fctl/xxx/.gitmodules'</p>