Let’s talk

Insights

How to run your first managed DevSecOps service review

A practical first-month review for GCC technology leaders, using release records, customer journeys and security exceptions to decide what needs to improve.

Share this insight

A perspective worth sharing.

Your LinkedIn caption and article image are ready. Edit the caption, copy it, then paste it into your LinkedIn post.

Share on LinkedIn copies your caption and opens the article preview in a new tab. Paste the caption before publishing.

Share on LinkedIn

Conceptual software release panels show a failed deployment returning to a stable version beside a service-job application.
Conceptual illustration of a software release review and the business service it supports. AI-generated; not a real provider dashboard or client deployment. Bridges

Start with one application

Suppose a provider reports 30 production deployments at its first monthly meeting. Before interpreting that number, the technology director needs to see what the changes delivered and how much work went into fixing earlier releases. The report should also show what customers experienced when a release failed.

Bring one important application to the meeting and ask the provider to walk through its deployment history. Connect that record to incidents and the business process the application supports. Use the first month to establish a baseline you can both reproduce, then agree what to change next. It is too early to attribute an improvement to the provider if there is no comparable earlier record.

Use this review once the engagement is operating. If you are still selecting a provider, start with the managed DevSecOps buyer's evidence checklist.

Agree what the report counts

Choose a service with a named business owner and a manageable release history. Record the reporting window, time zone, production environment and components included. A customer portal and a quarterly payroll application should have separate baselines; their change patterns and business consequences differ.

Ask for a deployment export with release identifiers, production timestamps and links to the relevant change records. Agree how the team treats rollback deployments, configuration changes and database migrations before interpreting the totals. Exclude builds that never reached production. Keep the underlying records so both sides can reproduce the report.

If deployment records cannot yet be linked to incidents, name that gap and give someone a date to resolve it. Reconstructing a few critical releases can help the meeting; it does not make a partly reconstructed month a complete baseline. Note any changes to measurement rules alongside next month's results.

Read five measures together

DORA's current guide defines five software delivery measures. They are useful for examining one application's delivery over time:

Software delivery measures
MeasureWhat it records
Change lead timeTime from a code commit to production.
Deployment frequencyDeployments in a period, or time between deployments.
Failed deployment recovery timeTime to recover from a deployment that needs immediate intervention.
Change fail rateShare of deployments requiring immediate intervention.
Deployment rework rateShare of deployments that were unplanned because of a production incident.

Use a short written definition for each measure in the review pack. Ask the provider to open the records behind an unexpected result. For each average, check the included records and inspect the cases that took longest or caused the most disruption.

Keep acknowledgement time separate from recovery time. Answering an incident ticket does not establish that customers can use the application again. Also separate recovery from a failed deployment from recovery after an unrelated network or supplier incident. Both deserve attention, but combining them would change the meaning of the measure.

Work through one month's releases

Consider an illustrative facilities-maintenance company. The figures below are invented to explain the review; they are not Bridges client results or industry benchmarks.

Its portal accepts service requests and sends jobs to dispatchers. During a 28-day window, the team makes 30 production deployments: 24 planned changes and six unplanned deployments to address production incidents. Three deployments require immediate intervention. In this example, those three failures lead to the six remedial deployments; the remedial deployments themselves succeed.

The change fail rate is 3 ÷ 30, or 10%. The deployment rework rate is 6 ÷ 30, or 20%. The two percentages answer different questions and should not be added. One counts deployments that failed; the other counts deployments made in response to incidents. They are not two mutually exclusive categories of production work.

Recovery takes 18, 35 and 110 minutes for the three failed deployments, giving a median of 35 minutes. The longest incident still needs its own discussion. In this example, request confirmations stopped working during that incident, leaving dispatchers to check whether calls referred to existing jobs or new ones. The review should examine duplicate requests and customer follow-up as well as restoration time.

In this scenario, part of the 110-minute delay came from waiting for permission to roll back. Agree who can authorise the rollback and assign someone to rehearse the decision with the duty team by a set date. Record whether the rehearsal actually reduced the wait; do not claim that it improved production recovery until there is evidence.

Without a comparison period, these figures cannot establish how well the provider is performing. Three failures also tell us little about future incidents. The approval delay is a specific problem the team can investigate now. If a later month has no failed deployments, record recovery time as having no observations, rather than reporting zero minutes.

Check whether customers could complete the job

The portal's business owner cares whether a service request reaches dispatch with a usable reference. A healthy web server can still sit behind a broken confirmation flow. Include a measure of that customer journey alongside the release report.

Agree what counts as a successful request, the time allowed and how the result will be checked. Define how retries and duplicate submissions are handled. If monitoring covers only the first screen, state that limitation instead of labelling the whole journey available.

A service level objective, or SLO, sets a reliability target. Google's guidance on implementing SLOs explains how an agreed target can help prioritise engineering work. For this portal, the business owner and provider could agree a target for accepted requests reaching dispatch within a specified time, then use actual journey records to review performance.

The error budget is the amount of failure permitted by that target over its measurement window. Agree what happens when it is used up. Google's example error-budget policy illustrates restricting changes while allowing urgent and security fixes. Decide who can invoke your policy, when normal releases can resume and who resolves disagreements. Choose thresholds for your business and agree the consequences before using them in the review.

Check contractual service levels and remedies separately from the operational target. A monthly technical discussion does not amend the contract.

Follow security exceptions to closure

Ask to see the issues still open against the application, including any accepted exceptions. For each material issue, check the affected release and exposure, then look at its owner and due date. Review old, serious issues even when the overall count is falling.

The NIST Secure Software Development Framework, version 1.1, published in February 2022, provides a shared vocabulary for discussing secure development with suppliers. Use it as a reference, not as a GCC-wide legal requirement. Check the service's actual contractual and jurisdiction-specific obligations.

Choose one issue marked closed and inspect the evidence supporting closure. Did the fix reach the production version in scope? Was it checked? If a risk was accepted temporarily, who accepted it and when must the decision be revisited? These are practical review questions; they do not establish certification or legal compliance.

Share evidence through agreed access controls. Avoid filling the review pack with customer records, credentials or unnecessary production log contents. Use redacted references where they still let the reviewer trace the decision.

Leave with a short decision record

Send the evidence pack before the meeting. Invite the business owner, the provider's delivery lead and the people responsible for operations and security, with roles combined where appropriate. Give them this agenda:

Monthly service review agenda
Review itemEvidence to openDecision to record
Reporting scopeApplication, dates and deployment exportAccept the baseline or assign missing instrumentation.
Release qualityFailed changes and remedial deploymentsSelect a recurring cause for investigation.
RecoveryThe longest failed-release timelineRemove a specific delay and name a rehearsal owner.
Customer journeyRequests, confirmations and dispatch recordsAgree the reliability target and response to a breach.
SecurityOpen issues and a sampled closureConfirm the remediation date or revisit risk acceptance.

For the illustrative portal, the decision record might prioritise the rollback approval rehearsal and a check that a request actually reaches dispatch. Give each action a named owner, completion date and evidence of completion. If work needs extra budget or changes the provider's scope, record who can approve it and keep that commercial decision visible.

Open the next meeting by checking those actions. Read the revised procedure alongside the rehearsal record to see whether the duty team could follow it. A later production incident may provide further evidence; a quiet month alone cannot prove that the weakness has been removed.

Set the review up for your operating team

Keep the review focused on the application and the actions you can fund. Ask the provider to distinguish observed results from missing measurements and show progress against those actions at the next meeting.

For UAE and Saudi operating teams, make the practical handovers explicit: working hours, time zones, after-hours authority and the language needed by the duty team. Specify where review evidence can be stored and who can access it under the organisation's own requirements. The applicable requirements depend on your organisation and jurisdiction.

To discuss delivery and operations scope with Bridges, see our managed IT, cloud and DevSecOps service. If the unresolved question is who makes decisions during an incident, use the managed-service incident rehearsal guide to prepare that discussion.

Share this insight

Related perspectives & expertise