outage DC3.HAM (Colt Telecom)

Since today around 1 am we are experiencing connectivity issues in our colocation DC3.HAM. We have created a ticket to Colt Telekom after initial investigation, since then we are pushing Colt Telekom. As of now we are in the 2nd highest escalation level and in the process of escalating to level 5 (VP at Colt). We seriously apologize for the inconvenience, be assured that we keep pushing. We have no explanation for this unbelievable bad service from Colt Telecom
update 09:35: escalation level 5 at Colt Telecom now.
the issue from our analysis is clear, ipv4 routing problem on Colt Telecom side
update 10:30: Colt Telecom has resolved the issue, DC3.HAM is reachable again
info: also our ticket system was affected, we apologize for late answers

Status Patch Day June 2024 – System Upgrades MCON Managed Services

Dear customer

According to our announcement below the patches for Region Europe, America and APAC are successfully done.

All systems are up and running. However if you encounter any problem please contact service@mcon-group.support.

Best regards,
Your MCON Managed Services Team

 

**********************************************************

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, June 03rd 2024, Development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, June 05th 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, June 07th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, June 24th 2024,  Development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, June 26th 2024, Test/Stage starting at 15:00PM (UTC+1 timezone).
  • Thursday, June 27th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, June 17th 2024, Development starting at 06:00AM (UTC-5 timezone).
  • Wednesday, June 19th 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, June 21st 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team

Patch Day July 2024 – System Upgrades MCON Managed Services

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, July 01st 2024, Development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, July 03rd 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, July 05th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, July 08th 2024,  Development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, July 10th 2024, Test/Stage starting at 15:00PM (UTC+1 timezone).
  • Thursday, July 11th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, July 15th 2024, Development starting at 06:00AM (UTC-5 timezone).
  • Wednesday, July 17th 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, July 19th 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team

DC3HAM shared infrastructure services Issue on 29th May 2024

UPDATE 2024-05-30 13:50 CEST

We have verified that all systems are currently working as designed. Availability of the affected services should be back to normal. We will continue to closely monitor the situation.


UPDATE 2024-05-29 17:41 CEST

mitigation is still ongoing as large volumes have to be moved. we continue to work 24×7


Dear Customer,

We are experiencing temporary downtimes of shared infrastructure services like Gitlab, Harbor, Rancher. Customer environments are not affected directly however in some cases e.g. some deployments are not possible. investigations have been difficult but we are are working with highest priority on the mitigation of the problem.

Best regards
Your CONVOTIS Munich Managed Service Team

Patch Day June 2024 – System Upgrades MCON Managed Services

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, June 03rd 2024, Development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, June 05th 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, June 07th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, June 24th 2024,  Development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, June 26th 2024, Test/Stage starting at 15:00PM (UTC+1 timezone).
  • Thursday, June 27th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, June 17th 2024, Development starting at 06:00AM (UTC-5 timezone).
  • Wednesday, June 19th 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, June 21st 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team

 

Status Patch Day May 2024 – System Upgrades MCON Managed Services

Dear customer

According to our announcement below the patches for Region Europe, America and APAC are successfully done.

All systems are up and running. However if you encounter any problem please contact service@mcon-group.support.

Best regards,
Your MCON Managed Services Team

 

***************************************************************

 

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, May 06th 2024, Development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, May 08th 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, May 10th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, May 13th 2024,  Development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, May 15th 2024, Test/Stage starting at 15:00PM (UTC+1 timezone).
  • Thursday, May 16th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, May 20th 2024, Development starting at 06:00AM (UTC-5 timezone).
  • Wednesday, May 22nd 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, May 24th 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team

DC3HAM Network Maintenance on 13th June 2024 by Colt

Dear Customer,

Pease be informed about the following  scheduled network maintenance by our carrier Colt.

Colt intends to carry out internal maintenance within its network. This has been designated as ESSENTIAL. The nature of this work is to carry out fiber cable work. Work is required to perform infrastructure maintenance to ensure the continued integrity of the network.
During the maintenance window you will experience one period of outage, with maximum duration of 7 hours.

The maintenance is scheduled for :

Start End
2024-06-13 21:00 GMT (Greenwich Mean Time) 2024-06-14 04:00 GMT (Greenwich Mean Time)
2024-06-13 23:00 CEST (Central European Summer Time) 2024-06-14 06:00 CEST (Central European Summer Time)

Colt would like to apologize for any inconvenience caused by this maintenance.

We will test the systems after the maintenance window and inform you about the status.

Best regards
Your MCON Managed Service Team

Patch Day May 2024 – System Upgrades MCON Managed Services

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, May 06th 2024, Development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, May 08th 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, May 10th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, May 13th 2024,  Development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, May 15th 2024, Test/Stage starting at 15:00PM (UTC+1 timezone).
  • Thursday, May 16th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, May 20th 2024, Development starting at 06:00AM (UTC-5 timezone).
  • Wednesday, May 22nd 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, May 24th 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team

 

Status Patch Day April 2024 – System Upgrades MCON Managed Services

Dear customer

According to our announcement below the patches for Region Europe, America and APAC are successfully done.

All systems are up and running. However if you encounter any problem please contact service@mcon-group.support.

Best regards,
Your MCON Managed Services Team

 

***************************************************************

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, April 01st 2024, development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, April 03rd 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, April 05th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, April 08th 2024,  development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, April 10th 2024, QA/Stage starting at 15:00PM (UTC+1 timezone).
  • Friday, April 11th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, April 15th 2024, development starting at 16:00PM (UTC-5 timezone).
  • Wednesday, April 17th 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, April 19th 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team

Patch Day April 2024 – System Upgrades MCON Managed Services

Dear customer,

In order to keep our server systems updated and secure, we need to apply patches on a regular basis.

The next patch days are scheduled :

Region APAC
  • Monday, April 01st 2024, development starting at 16:00PM (UTC+8 timezone).
  • Wednesday, April 03rd 2024, QA/Stage starting at 22:00PM (UTC+8 timezone).
  • Friday, April 05th 2024 between 01:00AM and 05:00 AM (UTC+8 timezone): Production systems.
Region Europe
  • Monday, April 08th 2024,  development starting at 09:00 AM (UTC+1 timezone)
  • Wednesday, April 10th 2024, QA/Stage starting at 15:00PM (UTC+1 timezone).
  • Friday, April 11th 2024 between 22:00PM and 03:00 AM (UTC+1 timezone) : Production systems.
Region America
  • Monday, April 15th 2024, development starting at 16:00PM (UTC-5 timezone).
  • Wednesday, April 17th 2024, QA/Stage starting at 09:00AM (UTC-5 timezone).
  • Friday, April 19th 2024 between 03:00AM  and 06:00 AM (UTC-5 timezone): Production systems.

There may be short downtimes during service or when the operating system restarts.

Best regards,

Your MCON Managed Services Team