Important Update on ByAll-AWS Migration
Subscribe to Updates
Update
Good afternoon,
We wanted to make you aware that the issues we have been updating you on related to ByAll’s migration to Amazon Web Services remain minimal and are nearing resolution.
The most notable issues that were related to data file delivery delays and AccountView access, specifically using login-based Single Sign-on via Data Connect, have been resolved.
There are some financial institutions still experiencing connectivity issues.
ByAll is continuing to make progress and is seeing performance improvements; they will be providing us with further updates as available. Please see the message below with the current update from ByAll.
Thanks,
Orion Reconciliation Team
------------------------------
Dear Valued Client,
After the AWS migration for Morningstar ByAllAccounts, we observed specific Custodial Integrator data delivery, performance-related issues, and financial institution connectivity issues.
Update as of November 29:
Financial Institution Connectivity: We are experiencing financial institution connectivity issues, resolving these issues is our highest priority. A comprehensive list of affected financial institutions and their status of resolution is attached in the FAQ referenced below. You can continue to reference this link for up-to-date information.
Custodial Integrator Data Delivery: Customers with large imports have been staggering their data delivery runs. Today we resumed regular import schedules. Our team continues to monitor our system's capacity, uptime, and performance to ensure seamless operations.
Please see our FAQ for specific details on user action and account update error messaging.
(https://advisor.morningstar.com/Enterprise/VTC/BAAIncidentFAQ.pdf?utm_source=eloqua&utm_medium=email&utm_campaign=none&utm_content=50109)
We appreciate your patience as we work through these unanticipated migration-related issues. This will be our final daily update, you will receive ad-hoc communications as high-priority financial institution connections resolve.
Regards,
The Morningstar ByAllAccounts Team
Posted Nov 29, 2023 - 15:45 CST
Update
Good afternoon,
We wanted to make you aware that the issues we have been updating you on related to ByAll’s migration to Amazon Web Services are now minimal and nearing resolution.
The most notable issues that were related to data file delivery delays and AccountView access, specifically using login-based Single Sign-on via Data Connect, have been resolved.
There are some financial institutions still experiencing connectivity issues.
ByAll is continuing to make progress and is seeing performance improvements; they will be providing us with further updates as available. Please see the message below with the current update from ByAll.
Thanks,
Orion Reconciliation Team
________________________________
Dear Valued Client,
After the AWS migration for Morningstar ByAllAccounts, we observed specific Custodial Integrator data delivery, performance-related issues, and financial institution connectivity issues.
Update as of November 27:
· Financial Institution Connectivity: We are experiencing financial institution connectivity issues, resolving these issues is a high priority for our team. A comprehensive list of affected financial institutions and their status of resolution is attached in the FAQ referenced below.
· Custodial Integrator Data Delivery: The primary challenge is the completion of concurrent Custodial Integrator data deliveries during the peak traffic hours of 6:00 am to 10:00 am. This results in an amount of traffic that affects the success rate and running time of data deliveries. We have communicated individually to customers with large imports to stagger runs and working with them to run their data delivery. As expected, implementing staggered data imports has helped to stabilize the system, we expect that this process will continue through Tuesday, November 28. We expect regular import schedule to go back into effect on Wednesday, November 29.
· AccountView Performance: Regarding AccountView, we deployed fixes over the weekend to address performance issues experienced last week. We are currently requesting feedback from customers to ensure it has addressed the performance issues. A limited applicable number of customers were having issues accessing AccountView using Login-based Single Sign-on via Data Connect. A fix was deployed on the night of Wednesday, November 22, we have since monitored activity and confirmed the issue is resolved.
Our team continues to monitor our system's capacity, uptime, and performance to ensure seamless operations.
Please see our FAQ for specific details on user action and account update error messaging.
We appreciate your patience as we work through these unanticipated migration-related issues. As we continue to work to resolve these issues for you, we will provide our next status update tomorrow.
Regards,
The Morningstar ByAllAccounts Team
Posted Nov 27, 2023 - 14:27 CST
Update
Good afternoon,
We wanted to make you aware that the ByAll issues we have been updating you on are not yet fully resolved, although there has been significant progress made.
ByAll has resolved the data file delivery and we do not anticipate delays going forward.
The most notable outstanding issue is related to AccountView access, specifically using login-based Single Sign-on via Data Connect. A limited number of customers are still experiencing access issues. A fix is currently in process and is expected to be resolved with the next two business days.
ByAll is continuing to make progress and is seeing performance improvements; they will be providing us with further updates as available. Please see the message below with the current update from ByAll.
Thanks,
Orion Reconciliation Team
_______________________
Dear Valued Client,
After the AWS migration for Morningstar ByAllAccounts, we observed specific Custodial Integrator data delivery, performance-related issues, and financial institution connectivity issues.
• Update as of Nov. 21:
• AccountView Performance: Regarding AccountView, we deployed fixes over the weekend to address performance issues experienced last week. We are currently requesting feedback from customers to ensure it has addressed the performance issues. A limited applicable number of customers are currently having issues accessing AccountView using Login-based Single Sign-on via Data Connect. A fix is currently in process, we expect it to be deployed within the next two business days.
• Financial Institution Connectivity: We are experiencing financial institution connectivity issues, resolving these issues are a high priority for the team. A list of the impacted financial institutions are listed in the FAQ provided below.
• Custodial Integrator Data Delivery: The primary challenge is the completion of concurrent Custodial Integrator data deliveries during the peak traffic hours of 6:00 am to 10:00 am. This results in an amount of traffic that affects the success rate and running time of data deliveries. We have communicated individually to customers with large imports to stagger runs and working with them to run their data delivery. As expected, implementing staggered data imports has helped to stabilize the system, we expect that this process will continue through Friday, November 24. Fixes were put in place over the weekend to improve performance, we are continuing to monitor.
Our team continues to monitor our system's capacity, uptime, and performance to ensure seamless operations.
Please see our FAQ for specific details on user action and account update error messaging.
We appreciate your patience as we work through these unanticipated migration-related issues. As we continue to work to resolve these issues for you, we will provide a status update later today.
Regards,
The Morningstar ByAllAccounts Team
Posted Nov 21, 2023 - 15:08 CST
Update
Good morning,
We wanted to make you aware that the ByAll issues we have been updating you on are not yet fully resolved.
As a result of the migration issues, ByAll data files are still being received slower than usual. We anticipate files will come in throughout the day. Please continue to check the Reconciliation Status App to verify the status of your Data Reconciliation in the meantime. Also, please feel free to reach out to our SME Data Reconciliation team if you have further questions regarding the status of Data Reconciliation.
ByAll is continuing to make progress and is starting to see performance improvements; they will be providing us with further updates as available. Please see the message below with the current update from ByAll.
Thanks,
Orion Reconciliation Team
________________________________
Dear Valued Client,
After the AWS migration for Morningstar ByAllAccounts, we observed specific Custodial Integrator data delivery and performance-related issues.
Update as of Nov. 20: Custodial Integrator Data Delivery: The primary challenge is the completion of concurrent Custodial Integrator data deliveries during the peak traffic hours of 6:00 a.m. to 10:00 a.m. This results in an amount of traffic that affects the success rate and running time of data deliveries. We have communicated individually to customers with large imports to stagger runs and work with them to run their data delivery. As expected, implementing staggered data imports has helped stabilize the system, and we expect this process to continue through Friday, Nov. 24. Fixes were put in place over the weekend to improve performance, and we are monitoring this morning. We did experience a minor service outage on Nov. 20 at 8:00 a.m. ET, which has been resolved.
AccountView Performance: Regarding AccountView, we deployed fixes over the weekend to address performance issues experienced last week. We are requesting customer feedback to ensure it has addressed the performance issues.
Our team continues to monitor our system's capacity, uptime, and performance to ensure seamless operations. We are still cautiously implementing changes to address performance issues.
Please see our FAQ for specific details on user action and account update error messaging.
We appreciate your patience as we work through these unanticipated migration-related issues. As we continue to work to resolve these issues for you, we will provide a status update later today.
Regards,
The Morningstar ByAllAccounts Team
Posted Nov 20, 2023 - 09:09 CST
Update
Good morning,
We wanted to make you aware that the ByAll issues we have been updating you on this week are not yet fully resolved. Please see the message below with an update from ByAll.
As a result of the migration issues, ByAll data files are being received slower than usual. We anticipate files will come in throughout the day. Please continue to check the Reconciliation Status App to verify the status of your Data Reconciliation in the meantime. Also, please feel free to reach out to our SME Data Reconciliation team if you have further questions regarding the status of Data Reconciliation.
ByAll is continuing to make progress and is starting to see performance improvements; they will be providing us with further updates as available.
Thanks,
Orion Reconciliation Team
________________________________
Dear Valued Client,
After the AWS migration for Morningstar ByAllAccounts, we observed specific Custodial Integrator data delivery and performance-related issues.
Update as of Nov. 17: Custodial Integrator Data Delivery: The primary challenge is the completion of concurrent Custodial Integrator data deliveries during the peak traffic hours of 6:00 am to 10:00 am. This results in an amount of traffic that affects the success rate and running time of data deliveries. We have communicated individually to customers with large imports to stagger runs and working with them to run their data delivery. We expect that implementing staggered data imports will help stabilize the system.
AccountView Performance: Regarding AccountView, we acknowledge that there continues to be performance issues which we are actively addressing and aim to have in production by Monday. These issues are linked to specific queries causing system degradation at certain points throughout the day.
Please see our FAQ for specific details on user action and account update error messaging.
We appreciate your patience as we work through these unanticipated migration-related issues. As we continue to work to resolve these issues for you, we will provide a status update this afternoon.
Regards,
The Morningstar ByAllAccounts Team
Posted Nov 17, 2023 - 10:11 CST
Monitoring
Good morning,
We wanted to make you aware that the ByAll issues we have been updating you on this week are not yet fully resolved. Please see the message below with an update from ByAll.
As a result of the migration issues, ByAll data files are being received slower than usual. We anticipate files will come in throughout the day. Please continue to check the Reconciliation Status App to verify the status of your Data Reconciliation in the meantime. Also, please feel free to reach out to our SME Data Reconciliation team if you have further questions regarding the status of Data Reconciliation.
ByAll is continuing to make progress and is starting to see performance improvements; they will be providing us with further updates as available.
Thanks,
Orion Reconciliation Team
________________________________
Dear Valued Client,
After the AWS migration for Morningstar ByAllAccounts, we had observed specific performance related issues.
Update as of Nov. 15: We are currently experiencing intermittent performance and accessibility issues, with some customers reporting 504 errors. Slowness was experienced during peak load times, which is primarily between 6:00 a.m. to 10:00 a.m. ET. We increased system capacity yesterday, we are continuing to work on the performance of database queries that are causing a backlog of requests.
Our team continues to monitor our system's capacity, uptime, and performance to ensure seamless operations.
Please see our FAQ for specific details on user action and account update error messaging.
We appreciate your patience as we work through these unanticipated migration-related issues. As we continue to work to resolve these issues for you, we will provide a status update later today.
Regards,
The Morningstar ByAllAccounts Team
Posted Nov 16, 2023 - 11:16 CST
This incident affects: Orion Internal Systems (Orion Case Management System, Orion Phone System, Orion Email), Platforms (Orion Connect, Advisorlynx.com, Mobile App, Client Portal, Reporting, Citrix / Advisor Desktop, www.OrionAdvisor.com, Orion Social, TOM, Eclipse, FIX Trading, ASTRO, Financial Planning), and Server Instances (Instance 1, Instance 2, Instance 10, Instance 11, Instance 12, Instance 13, Instance 14, Instance 15, Instance 16, Instance 20, Instance 21, Instance 22, Instance 23, Instance 3, Instance 24, Instance 25, Instance 26, Instance 27, Instance 7005, Instance 7010, Instance 7018, Instance 7023, Instance 7025, Instance 7026, Instance 7015, Instance 7016).