Category Archives: Devices

Projekt Cumulus: Was tun bei IOT Rollout Problemen?

Posted on by
A group of four professionals in a darkened room, collaborating around a table filled with electronic devices. They are analyzing data displayed on multiple computer screens, with one screen showing an 'INCIDENT ALERT'.

IoT Rollout-Incidents als Workflow: kontrollierte Changes statt „wir drehen schnell zurück“

Firmware- und Konfigurations-Rollouts sind unvermeidlich. Ebenso unvermeidlich ist, dass gelegentlich etwas schiefgeht: Fehlerquoten steigen, Geräte melden neue Error Codes, Regionen sind unterschiedlich betroffen.

In der Praxis entsteht dann oft Hektik: Stoppen? Zurückrollen? Canary-Deployments? Wer informiert wen? Und wie stellt man sicher, dass Entscheidungen sauber begründet und später nachvollziehbar sind?

Cumulus setzt deshalb auf ein Prinzip: Agenten helfen beim Verstehen und Planen, aber die Ausführung bleibt kontrolliert (kleine Steps, Regeln, begrenzte Fehlerpfade). So wird Incident Response bei Rollouts schnell – ohne Chaos.

Kurzstory

Ein Rollout läuft seit zwei Stunden. Plötzlich steigt in einer Region die Fehlerrate, Support bekommt die ersten Tickets, und Engineering sieht widersprüchliche Signale: Bei einem Modell ist alles normal, bei einem anderen häufen sich neue Error Codes. Die Frage im Raum: „Pause? Rollback? Canary Deployment enger?“ – und wer informiert welche Stakeholder?

Ein Rollout-Incident-Workflow sorgt dafür, dass die ersten Schritte standardisiert sind: Scope wird segmentiert, Severity wird nach klaren Kriterien bestimmt, Maßnahmen laufen über definierte Gates, und Kommunikation ist konsistent – damit schnelle Entscheidungen später auch nachvollziehbar bleiben.

1) Warum Rollout-Incidents mehr als „Rollback drücken“ sind

Ein Rollout-Incident ist selten eindeutig. Typische Fragen:

  • Betrifft es alle Geräte oder nur ein Segment (Region, Modell, Firmware-Baseline)?
  • Ist es ein echter Incident oder ein Mess-/Telemetry-Artefakt?
  • Was ist die richtige Maßnahme: Pause, Rollback, Hotfix, Canary erweitern?
  • Welche „Guards“ gelten (Sicherheit, Verfügbarkeit, Kundenimpact)?
  • Welche Kommunikation ist nötig (Support, Customer Success, ggf. Kunden)?

Das ist ein Prozess mit Entscheidungspunkten – und damit ein Workflow.

2) Agentische Herkunft: Der Agent als Plan-Autor

Agenten sind stark bei:

  • Verdichten von Signalen zu einer Incident-Zusammenfassung
  • Vorschlägen zur Segmentierung („Was ist gemeinsam bei den betroffenen Geräten?“)
  • Entwurf von Next Steps (Checks, Hypothesen, Kommunikationsentwürfe)

Wichtig: Agentik liefert Vorschläge. Der Rollout selbst wird über kontrollierte Steps geändert.

3) Steps: Änderungen in kontrollierbaren Einheiten

Statt „wir reagieren irgendwie“ wird der Incident in Steps aufgeteilt:

  • Incident verstehen & scope bestimmen
  • Evidence sammeln (Metriken, Error Codes, betroffene Segmente)
  • Entscheidungspfad anwenden (Pause/Rollback/Canary)
  • Kommunikation auslösen
  • Stabilisierung & Nachbereitung dokumentieren

So bleibt das Team handlungsfähig, auch wenn mehrere Stakeholder parallel arbeiten.

4) Agentic vs. Logic: Analyse vs. sichere Rollout-Aktionen

  • Agentische Steps: Hypothesen, Segmentierungs-Ideen, Textentwürfe.
  • Deterministische Steps: „Gates“ (Stop/Go), Standardmaßnahmen, Eskalationslogik, Status-Updates.

Damit trifft man Entscheidungen schnell, aber nicht willkürlich.

5) Beispiel-Blueprint: Rollout-Incident in 7–9 Schritten

  1. Incident-Trigger erfassen (Error-Rate/Device Reports) und zusammenfassen
  2. Scope segmentieren (Region/Modell/Version/Update-Welle)
  3. Severity bestimmen (Sicherheit/Verfügbarkeit/Kundenimpact)
  4. Sofortmaßnahme wählen: Pause, Canary zurücksetzen, Rollback vorbereiten
  5. Evidence sammeln: Repro, Logs/Signals, Vergleich mit Kontrollgruppe
  6. Entscheidung treffen (Rollback vs. Hotfix vs. Canary-Adjust) – mit klaren Regeln
  7. Kommunikation: interne Stakeholder + ggf. externe Info
  8. Verifikation: KPIs wieder im Normalbereich?
  9. Postmortem: Guardrails verbessern (Canary, Checks, Policies)

6) Regelwerk statt “im Druck entscheiden”

Ein stabiler Incident-Workflow braucht klare Regeln:

  • Stop/Go-Gates je Severity
  • klare Rollen (Incident Commander, Kommunikation, Engineering Owner)
  • definierte Maßnahmenkataloge (Pause, Rollback, Canary enger)
  • begrenzte Fehlerpfade (fehlende Evidence → gezielte Datenerhebung)

Das reduziert die Zeit bis zur Stabilisierung und erhöht die Qualität der Nachbereitung.

7) Repair & Human-in-the-loop: kontrollierte Unsicherheit

Auch hier sind Inputs oft unvollständig. Cumulus unterstützt:

  • Repair-then-Retry für unklare Segmentierung oder widersprüchliche Hypothesen
  • Human-in-the-loop für riskante Aktionen (z. B. flächiger Rollback, sicherheitsrelevante Entscheidungen)

Kurzablauf als Diagramm (high level)

A flowchart illustrating a process involving rollout signals, an incident plan created by an agent, controlled execution of steps by an engine, rules/gates, stabilization and postmortem, repair actions, and human-in-the-loop decision-making.

Für Ops/Leitung: Was sich messbar verbessert

  • Time-to-Stabilize sinkt: Der Incident startet mit einem klaren Plan statt mit Ad-hoc-Diskussionen.
  • Kleinere Blast Radius: Gates/Regeln reduzieren die Ausbreitung von fehlerhaften Changes.
  • Bessere Kommunikation: Stakeholder erhalten konsistente Updates, nicht widersprüchliche Zwischenstände.
  • Wiederverwendbare Postmortems: Outcomes führen zu verbesserten Guardrails und Playbooks.
  • Mehr Vertrauen in Rollouts: Änderungen bleiben kontrolliert – auch unter Druck.

Fazit

Rollout-Incidents sind kein Einzelfall – sie sind ein wiederkehrender Prozess. Cumulus macht daraus einen Workflow:

  • agentische Planung für schnelle Orientierung
  • kontrollierte Ausführung über Steps, Regeln und Gates
  • nachvollziehbare Entscheidungen und saubere Kommunikation

So bleiben Sie schnell – und gleichzeitig professionell im Incident Management.

Nächster Schritt

Wenn Sie möchten, können wir aus Ihrem Rollout-Prozess (Canary-Deployment-Strategie, Verantwortlichkeiten, KPIs) ein konkretes Template ableiten, das Sie für jeden Release wiederverwenden können.

Rising Above the Cloud: Turning In-House Giants into Cloud Innovations

Posted on by

It was a great pleasure to talk with AVNET SILICA’s host Stefanie Heyduck about the benefits of Cloud-based data collection and analysis in R&D scenarios.


We discussed how to raise the potential of any file-based data by freeing it from its shackles using Cloud performance, functionality and scale. Getting rid of limiting file boundaries will expand your vision of the Universe.

“WE TALK IOT” certainly is one of the best IOT information sources currently available. Always at the pulse of time.
 

Thanks for having me!
Alexander

Azure IOTHub – time to update endpoint filter

Posted on by

There is a new and more secure IP-endpoint-filter out! The new one is more secure and the old one will be retired after a while. So, if you are using IOTHub in your solution, you need to upgrade!

Nevertheless, test the filter in your development environment, before rolling it out in production.
It should normally not have any negative effects, if you are using the endpoints in a standard way, but one only ever knows after trying it out.
🙂

Alexander

Azure Digital Twins reach Production

Posted on by

Azure Digital Twins are the virtual counterparts of systems, sensors or even complete factories in the real world.
The Digital-Twin concept has already around for a time and I have used it in several customer projects, to get a as-good-as real-time view on the state of complex systems. It comes with the additional benefit of having historical data, e.g. to follow up on errors, or predict the future with the help of machine learning algorithms. In addition, the ability to simulate and test possible future situations or different development scenarios with a close-to-reality model, cannot be overrated!

While these custom implementations are working great, it must be admitted that there is significant effort necessary to reach this goal.
Due to this, I consider Azure Digital Twins as the arrival of a game changing platform service for future IOT solutions. Azure Digital Twins save a lot of development effort, are very good integrated with other Azure IOT offerings such as IOTHub, IOT Central and build on IOT Plug & Play. This is taking the fast lane !
It is a powerful combination of services, which are going to revolutionize the way IOT solutions will be built in the coming years.
The good development story behind Twins is supported by great tools for visualizing and reporting. This is something often neglected by standard IOT approaches. Any neglection in this area is dangerous, because capable reporting and querying functionality is essential to run, maintain and evolve your solutions in field.

I predict the Azure Digital Twins will be seen quite often in upcoming solutions.
🙂

Alexander

Azure IOT Central – Updates

Posted on by

IOT Central is Microsoft’s low code, low effort, ease of use approach into the world of embedded projects. This is quite a demanding challenge, because real world problems tend to be complex and what can you do to make these simple in a tool?
Well, normally you start with defining an environment, to get rid at least of some of the parameters and thus reducing complexity. This is a valid approach, but for a tool/service vendor it carries the danger that the overlap of your defined environment to common real-world use cases of customers, is not large enough, or, as a worst case, even not existing.
Azure IOT Central, in the beginning, felt a bit like: great base features, but not enough to cover a complete project spectrum of demands.
Therefore, to me it was good for samples or a quick POC for a project. However, the IOT Central team kept improving steadily and so the product is getting more serious as we speak.

The newest update provides some very interesting features, like jobs that can be execute on devices (very important for device management), webhook improvements looking at identity management, device templates to support IOT Plug & Play as well as improvements on the dashboard.

At least for me enough new stuff to justify a closer and serious re-visiting look into IOT Central!

🙂
Alexander

IOT Projects and Azure Time Series Insight

Posted on by

In nearly every IOT project I had the opportunity to work in, time series data played a very important role.
The problem for this type of data is that it normally comes in larger volumes and is therefore not always great to handle. This is especially true in projects, where you have to cope with small storage on devices and no central data store, which makes it very hard, if nearly impossible to get a global view on the behavior of these solutions in time.
One could work with thresholds and alerts, but this approach never gives you the chance to detect trends and get “ahead of the wave” to react better, faster and more precise to certain events. Some of the industrial communication standards, such as OPC UA and SCADA, try to tackle this issue by providing historic data functionality in their communication layers, but this is just a single aspect of a comprehensive data solution.


Cloud architectures are able to help in this case, if you have the chance to collect time series data either centrally, or on the edge.
A very valuable asset in Azure and in this context is Time Series Insights. It is a cloud service allowing you to handle query, transform visualize and correlate your different data streams into comprehensive views and insights. There are also connectors into reporting tools such as Power BI available. Using the M365 infrastructure Power Automation or Azure Logic Apps and Functions, serverless integration into corporate business process and control processes is also not a problem.

Get some insight into Insights (sorry for the pun 🙂 ) in this new podcast by Diego Viso, the Time Series Insights Principal PM.

Alexander

Surface Duo

Posted on by

I worked a lot with Microsoft mobile devices during my professional career helping OEMs to create devices as well as supporting customers to operate and manage up to 40.000 Windows Phones in their companies.
The last version of Windows Lumia Phones had great hardware and they were really useful enterprise class devices, but, on the other hand, could not make an impact in the all-defining consumer market. This, mainly due to their lack of apps and small size of the eco-system.
It was a sad day for me, when Microsoft pulled the plug on their phone business and I had to stow away my Lumia 950 XL, which I really liked due to its high-class, razor-sharp OLED display and the Windows Phone tile UI, which was easy and direct to operate. App development with C#, Visual Studio and .NET was fun and deployments secure using e.g. SCCM or Intune.
Sorry, if this sounds a bit nostalgic! 🙂

However, I would never had thought that Microsoft would enter the mobile device space again after the huge losses the last attempts have created.

Surface Duo, therefore, was more than a surprise to me and in the beginning I was really skeptical, if Microsoft was having a “great idea” or just running another attempt to get a “bloody nose”!

After now having a closer look at the specs and capabilities, I cautiously tend to issue a “great idea” judgement, because Microsoft is doing quite some things differently this time!
They are not trying to create a new development platform, but are betting on Android, an operating system created by a competitor, which is quite a step for the company.
The obvious benefit is that immediately there is a wealth of apps and an intact eco-system available!
In addition, they have focused innovation a new device class, the book design, which remotely reminds me at devices with keyboard like the Nokia Communicator as well as some of the HTC Pocket PC models. But, this time the approach is much more versatile, leveraging the two touch screens as display as well as input devices using pen or on-screen keyboard.
The book design with hinges to me looks also much more robust and pragmatic than some of the folding screen approaches by the competition.
There is some ongoing discussion on the missing second camera, but for normal day use cases the hardware looks well-equipped enough.

Major pain points are the really high price, probably significantly over 1.300,00 € over here in Europe and the fact that the device is currently sold only in the US and foreign markets are treated as second or third class citizens.
Looking at the relatively short life time of mobile devices, this is hard to understand and companies such as Samsung and Apple, of course do global rollouts to surf the wave of excitement any device release creates within their dedicated user group.
Not to mention that the history of this approach is not so encouraging looking at the list of devices (Zune, Microsoft Band, etc., …) that never went successfully global after an America-First release.

To get more technical info on Surface Duo, have a look at the great video above, or read the interesting and detailed Microsoft Mechanics blog post, which, thankfully, dives into technical details, to spare you the superficial marketing bla-bla one finds nowadays on standard product pages in the store.

Will I buy one, as soon as it becomes available here in Germany?
Well, I am heavily tempted, because I do have a feeling that such a device could be a great productivity gain, kind of a small laptop at hand, especially travelling on plane or train, although I still think the price should be more reasonable!

However, sometimes there is pain, when you try to be “cutting edge”!


I’ll keep You posted! 🙂
Alexander