Choosing a managed services provider is not a commoditised decision. Many organisations approach it as if it were. They’ll compare response times, price per seat, and reference lists. These matter, but they’re not the real differentiator.
This article walks you through how to evaluate vendors using a framework that goes beyond service-desk performance. For the foundation, read Managed Services for Microsoft 365: Much More Than a Support Desk.
When shadow AI becomes a compliance problem
One engagement illustrates the approach. A professional services firm in South Africa operates a franchise model across multiple provinces. Its CEO and head of IT scheduled a conversation with Braintree and Microsoft about artificial intelligence adoption.
That was the stated agenda. What emerged in the first discovery conversation was a different problem entirely. The organisation had no approved, centrally governed approach to AI. Employees were using personal versions of tools such as Claude and Gemini without organisational oversight.
For a professional services firm handling sensitive client data, this created compliance exposure. Business data was entering AI services outside approved processes. This was shadow AI at scale.
What changed
The managed services approach addressed four specific areas:
- Copilot implementation. The organisation moved to Microsoft Copilot in Microsoft 365, formerly Microsoft 365 Copilot, as its approved AI platform. This created a basis for applying Microsoft 365 permissions and audit controls. Copilot does not mean that every query stays inside a tenant: administrators still need to assess the processing, terms, and controls for enabled models, web search, and agents.
- Licence optimisation. In this engagement, analysis of licensing and purchasing patterns identified a Copilot procurement option at a 50% discount against the standard price used in the comparison. That was specific to the purchase and terms available at the time, not a standing offer or a saving every customer can expect.
- Single service desk. Instead of managing fragmented toolsets, the organisation had one contact point for AI and Microsoft 365 support, with improved response times.
- Data strategy roadmap. A data cleansing roadmap that finally includes POPIA and client confidentiality requirements. This has become the foundation for sustainable AI adoption beyond the initial rollout.
The results
In this engagement, the pilot of 30 Copilot users expanded to 60 users within two months. Leadership saw strategic value, cost savings, and a clearer approach to risk. The IT manager had a defined route for addressing shadow AI and data loss prevention, rather than a claim that all risk had disappeared.
The deeper value was clarity. The company now has a documented 12-month roadmap for AI adoption, data governance, and training, all under a managed services engagement. They know what they’re paying for, what they’re getting, and what the next phase looks like.
This is managed services as it should be.
What a CFO should measure
Once you’ve engaged a managed services provider, reporting should be consistent and specific. Here’s what to demand monthly or quarterly:
Licence optimisation metrics
- Cost reduction from baseline. What percentage of your licensing spend has been optimised? Our experience includes first-quarter licence savings of 20 to 40 percent where the starting position permits them, as discussed in the companion article. The achievable saving depends on your licences, usage, and contract terms. By month six, reporting should show realised savings and any further opportunities.
- FinOps variance analysis. What did the provider charge you vs. KPIs delivered? This transforms cost from a mystery into a managed line item. You should see invoice reconciliation and month-to-month variance explained.
Security metrics
- Secure Score trajectory. You should receive specific recommendations to accept or decline, with quarterly visibility into what changed and why. Track both the score and the underlying controls; a higher score alone does not establish compliance or eliminate risk.
- Risk surface reduction. How many critical vulnerabilities were closed? How many access points were tightened? This is qualitative but measurable through audit logs and configuration reviews.
Enablement metrics
- Training completion rates. What percentage of your user base completed required modules? Track this by role or department so you can see where adoption is strongest and where support is needed.
- Application adoption. What percentage of licensed features are actually being used? If you’ve licensed Power BI or Project, are users accessing it? Are they generating reports?
- Post-training usage lift. Did training change behaviour? Compare usage patterns before and after training to see if people are actually using what they’ve learned.
Strategic metrics
- AI governance. Unapproved AI use identified and addressed? Approval workflow in place? Audit trails active and reviewed? The engagement should show measurable progress in how AI use is governed.
- Compliance status. POPIA safeguards documented? Technology and information governance addressed under the applicable King Code? Board-level understanding of security posture established? King V supersedes King IV for financial years beginning on or after 1 January 2026. Reporting should reflect the organisation’s applicable requirements.
- Vendor accountability. Is C-suite reporting happening (CFO, CIO, CEO attendance)? Monthly or quarterly? Are conversations focused on progress against these metrics, not ticket counts?
If your provider is sending you a monthly report showing “47 tickets closed, 95% resolved within SLA,” you may be getting a decent service desk, but that isn’t managed services.
The vendor checklist for managed services
If you’re evaluating managed services providers, ask these questions upfront.
- What’s your licence optimisation method? Ask for relevant examples, the baseline, and how savings were calculated. What’s their methodology, and how long does it take? Evidence matters more than a headline percentage.
- How do you handle Microsoft pricing and forex volatility? This matters in South Africa. Ask which currency, commitment term, renewal date, and price-change rules apply to your agreement. Do they monitor Microsoft announcements, explain the effect on your costs, and advise on purchase timing? Where exchange rates affect your bill, how is that exposure managed?
- Show me your Secure Score approach. What’s your methodology for improving Secure Score? What target fits our environment? How do you report progress? Can you show a customer’s starting point, actions taken, and current position? Ask how they prioritise actual risk alongside the score.
- What training and enablement do you include? Don’t accept vague answers like “we provide training.” Ask: What modules? How long is the engagement? How do you measure adoption?
- How do you handle AI governance? Ask: Do you identify shadow AI? Do you have an approved platform recommendation? If they haven’t thought through AI, they’re not future-proofing your business.
- What’s your B-BBEE status? Ask for current evidence and confirm how it is recognised under the codes and procurement criteria applicable to your organisation. Establish this upfront.
- What does compliance reporting look like? Do you understand POPIA and the applicable King Code? Can you demonstrate the relevant controls and governance processes? Ask for examples of how you have helped other organisations address their requirements.
- Do you offer customised plans or a standard model? Braintree uses a standard approach that’s working well in South Africa right now. Larger enterprises may need customisation. Ask what their approach is. There’s no wrong answer here, but you need to know whether you’re getting flexibility or a one-size-fits-all contract.
- What’s your reporting cadence? Monthly? Quarterly? Who attends (CFO, CIO, CEO)? What metrics do you cover? If they can’t clearly describe how often you’ll see reporting and what’s included, that’s a red flag.
- What’s your investment in AI, automation, and compliance? This is a culture question. Are they actively investing in these areas, or are they still “crawling under desks pulling wires”? Ask about their product roadmap and capability investments. You want a partner who’s moving forward, not standing still.
The South African competitive landscape
When reviewing local managed services providers, look beyond SLA-based support and basic licence optimisation. Check for these common gaps:
- Security-first approach. Is there a clear Secure Score methodology and security governance, or does the provider rely on isolated security projects?
- AI strategy. Is shadow AI being addressed? Is there a Copilot implementation framework and a governance model?
- Compliance architecture. Are POPIA safeguards and applicable King Code governance built into the engagement?
- Long-term roadmap. Is there a strategic plan beyond individual support transactions?
This isn’t to say other providers can’t deliver. It’s to say: ask the vendor checklist above. The answers will show you who’s serious about managed services and who’s repackaging old models.
The BEE factor
Your managed services provider’s B-BBEE status can affect the procurement recognition available to your organisation. The result depends on the applicable codes, your scorecard, and the procurement criteria; the supplier’s level alone does not determine your final score.
Ask every vendor for valid evidence of its status, such as a verification certificate or an eligible sworn affidavit. Check its validity and relevance to your procurement requirements before appointment.
The first 90 days set the tone for the entire engagement
Once you’ve selected a vendor, the first quarter is critical. You should see something like this:
Month 1: security assessment and licence optimisation audit
Your vendor documents your baseline Secure Score. They map your current licence spend. A discovery report shows where you stand across the five focus areas. You walk away from month one knowing your starting point and what’s possible.
Month 2: recommendations and roadmap
Your vendor prioritises security improvements, both quick wins (like enabling MFA) and longer-term work. They deliver a licence optimisation roadmap showing where savings will come from and when you’ll realise them. They build a training plan with modules, schedules, and success metrics.
Month 3: implementation begins
Security improvements go live first. Licence optimisation executes in parallel. Enablement (training) launches. By the end of month three, you see tangible progress: Secure Score has moved, you’ve consolidated licences, and your initial training cohorts have completed their first modules.
If you’d like to discuss whether your current managed services is delivering on these outcomes, contact Braintree to arrange a review of your situation.
