Copado, a leading DevOps platform powering some of the world’s largest digital transformations, today released the findings from its “2022 State of Salesforce Testing” report. The first of its kind survey of nearly 300 Salesforce executives, developers and quality assurance professionals collected hundreds of data points to provide insight into the software testing practices of the most sophisticated companies using Salesforce.

Testing in Salesforce becomes more critical as the ecosystem of custom applications, managed packages from the AppExchange and integrations are involved. Not only do organizations have to adapt to changes when it comes to their Salesforce deployments, but they also have to track integrations with other systems that have their own release cycles. With this rapid innovation comes the need for scalable end-to-end testing. A recent Forrester report confirms organizations using low-code platforms like Salesforce can’t afford to ignore automated testing.

The Copado State of Salesforce Testing report underscores the challenges organizations are facing as they incorporate testing into their release cycle. Key takeaways include:

Quality is more than a full-time job

 Organizations employ a significant amount of resources toward software testing:

• 49% of organizations employ a full-time testing team; 26% outsource this function to a consulting company; 23% rely on developers to handle code testing.
• 51% of organizations have more than 26 resources dedicated to quality assurance full time; 20% employ more than 100 professionals tasked with this responsibility.

 

Manual testing isn’t scalable — and is expensive

 Testing remains the bottleneck to digital transformation. Although technology continues to advance, companies are still engaged in traditional testing methods such as manual testing or script-based solutions. The consequence is the quality of releases suffers since there isn’t enough time to test changes before deployments. Defects found later in the release cycle are exponentially more expensive to fix. Therefore production defects can have a massive financial impact on companies:

• 41% of organizations don’t have enough time to fully test all changes before a release due to time constraints that force teams to reduce the scope of testing.
• 92% experience production issues (defects uncovered) each year due to inadequate testing.
• 84% of respondents still rely at least partially on manual testing. 30% of companies use home-grown testing scripts, typically based on Selenium, while 35% employ commercial testing tools.
• Manual testing has a 95% higher total cost of ownership than using commercial test automation tools.

 

Automated testing can dramatically improve outcomes

The cost of not adopting automated testing is a much higher incidence of production failures despite greater investment in the testing team:

• Teams using automated testing experienced less than half as many production failures per year as a result of releases (a median of 3 per year instead of 7).
• Teams using automated testing release 50% more frequently than those relying on manual testing (34 times per year on average, compared to 22 times per year).
• Teams using automated testing were 50% more likely to complete all of their test plans for each release (67% of teams using automated testing reported completing all their tests for each release vs. only 45% of teams relying on manual testing).

“Companies are clearly finding that automated testing is the best way to move fast and not break things,” said Pat McQueen, Senior Vice President of Growth at Copado. “Teams that prioritize their testing strategy outperform those who don’t. The investment in automated testing enables business agility which leads to long-term commercial success in a competitive market.”

The full report can be found at: https://www.copado.com/salesforce-testing

 

Methodology

Copado surveyed over 275 executives, managers and members of Salesforce development teams to learn about their testing process. Responses came from a variety of industries, but the majority were from technology, healthcare and pharmaceuticals, industrials and manufacturing, and financial services. The majority of respondents were Salesforce developers and quality assurance professionals, but a wide range of roles were represented. Most respondents were from companies with more than 1,000 employees. A large number were from global system integrators who may employ hundreds of thousands of people. The goal of this analysis was to understand the challenges and successes teams face in testing Salesforce. Non-trivial analysis was done on the Tableau Analytics platform to enable data visualization, cleansing, and cross tab analysis.