Should Your Best Reps or Worst Reps Run the Pilot

When launching an AI pilot for your sales team, one of the first strategic decisions is who will participate. Should you select your top-performing reps, your struggling reps, or a mix? The answer depends entirely on the primary goal of your pilot.

If your goal is to prove maximum potential and generate compelling ROI numbers quickly, then your best reps are the logical choice. If your goal is to identify workflow friction, uncover training needs, and understand the tool’s baseline usability, then including struggling reps provides more valuable insights.

Aligning Participant Selection with Pilot Goals

Before selecting any rep, define what success looks like for this specific pilot. Are you trying to validate a vendor’s claims, optimize a specific part of the sales cycle, or simply understand how a new AI tool integrates into daily workflows? Your objective dictates the ideal participant profile.

Pilot Goal: Maximize ROI and Prove Value

If your main objective is to generate the highest possible ROI numbers to secure further investment or wider adoption, focus on your top performers. These reps already have strong sales fundamentals and are likely to adapt quickly to new tools. They can leverage the AI to enhance their existing strengths, leading to impressive initial results.

“The fastest path to a compelling ROI number in a pilot is often through your most effective users.”

Top performers are also more likely to be early adopters and internal champions. Their positive experience can create momentum and reduce resistance from other team members when the tool scales. They can articulate the benefits clearly, drawing on their own success.

Pilot Goal: Identify Friction and Usability Issues

If your goal is to stress-test the AI tool, uncover workflow bottlenecks, and understand its true usability for the average rep, then include struggling or average performers. These reps will expose areas where the AI tool is not intuitive, where training is insufficient, or where the integration with existing processes breaks down.

This approach provides a more realistic assessment of the tool’s general applicability. It helps you refine the implementation strategy, develop better training materials, and identify necessary adjustments before a full rollout. Ignoring these insights can lead to widespread adoption issues later.

The Case for Top Performers

Using your best reps in an AI pilot offers several advantages. They are often more motivated, tech-savvy, and adept at incorporating new processes.

Advantages of Top Performers

• Higher Adoption Rates: Top reps are often more open to experimenting with new tools that can give them an edge. They are less likely to abandon the tool due to initial friction.
• Stronger Results: Their existing skills mean they can quickly integrate the AI tool into their workflow and achieve measurable improvements, leading to impressive pilot metrics.
• Internal Advocacy: Successful top performers become powerful advocates, influencing their peers and leadership. Their positive testimonials can drive wider team adoption.
• Clearer Feedback on Value: They can articulate how the AI tool directly impacts their most effective strategies, providing insights into its true value proposition.

Potential Drawbacks

While beneficial for proving ROI, relying solely on top performers can create a skewed view. The tool might appear more effective than it would be for the broader team. It might also mask usability issues that average reps would encounter, leading to a difficult rollout later.

The Case for Struggling or Average Performers

Including reps who are not top performers provides a different, but equally valuable, set of insights. They represent the majority of your sales force and can highlight areas for improvement in the tool or its implementation.

Advantages of Struggling/Average Performers

• Exposing Usability Gaps: These reps are more likely to struggle with complex interfaces or unclear instructions, revealing critical areas where the AI tool needs simplification or better training.
• Identifying Training Needs: Their challenges highlight specific knowledge or skill gaps that need to be addressed in training programs. This helps create more robust onboarding for the entire team.
• Realistic Baseline: Their performance provides a more accurate baseline for what the average rep can expect from the AI tool, preventing inflated expectations.
• Workflow Integration Insights: They can uncover where the AI tool disrupts existing workflows or fails to integrate smoothly, prompting necessary process adjustments.

Potential Drawbacks

The main risk here is that the pilot might show lower ROI or even negative results if the tool is not well-suited for their needs or if training is inadequate. This could lead to a perception that the AI tool is ineffective, even if it has potential. It also requires more patience and support from pilot managers.

A Mixed Approach: Balancing Perspectives

A balanced approach involves selecting a small, diverse group of reps. This could include one or two top performers, a few average performers, and perhaps one struggling rep. This strategy aims to capture both the peak potential and the common challenges.

Benefits of a Mixed Group

• Comprehensive Feedback: You get insights into both high-performance scenarios and areas needing improvement.
• Broader Applicability: The results are more representative of the entire sales organization, making it easier to predict company-wide impact.
• Peer Learning: Top performers can mentor others, and the group can collectively identify best practices for using the AI tool.

Challenges of a Mixed Group

• Confounding Results: It can be harder to isolate the impact of the AI tool when performance varies widely among participants. Clear metrics for each segment are crucial.
• Increased Management Overhead: Managing diverse skill levels requires more tailored support and attention from the pilot lead.

When considering how many reps should be in an AI pilot, a mixed group might require a slightly larger pilot size to ensure statistical significance across different segments.

Pilot Design Considerations

Regardless of your participant selection, certain design elements are critical for a successful pilot.

Clear Objectives and Metrics

Define specific, measurable, achievable, relevant, and time-bound (SMART) objectives for your pilot. What specific KPIs are you trying to move? Examples include:

• Increase conversion rates by X%
• Reduce time spent on Y task by Z hours per week
• Improve lead qualification accuracy by A%

Training and Support

Provide comprehensive training tailored to the AI tool and ongoing support. This is especially critical if you include struggling or average performers. Ensure there’s a clear channel for feedback and troubleshooting.

Regular Check-ins and Feedback Loops

Schedule frequent check-ins with pilot participants. Gather qualitative feedback on usability, integration, and perceived value. This helps you iterate quickly and address issues before they derail the pilot. If you find yourself asking what to do when an AI pilot produces no clear result, it often points back to unclear objectives or insufficient feedback loops.

Data Collection and Analysis

Establish robust data collection mechanisms to track the defined KPIs. Compare pilot group performance against a control group or historical data. This quantitative analysis is essential for proving ROI.

Iteration and Adaptation

Be prepared to make adjustments based on pilot findings. This might involve refining the tool’s configuration, updating training materials, or even pivoting the pilot’s focus. Sometimes, you might need to restart a stalled AI pilot if initial assumptions were incorrect.

Decision Matrix for Participant Selection

This table summarizes the participant selection strategies based on pilot goals.

Ultimately, the decision of who should run your AI pilot is a strategic one. It should align with your immediate goals for the pilot phase. A well-designed pilot, regardless of participant selection, requires clear objectives, strong support, and a commitment to data-driven decision-making.