Our survey engine is very customizable, and changing the number of questions sent out in each mailing is easy. Changing the mailing frequency is usually not a problem, but changing it to extreme values and sending out many more questions than we recommend comes with a price. Let's review how the system works and what could happen if you ask too many questions.
How our AI works
Winningtemp's algorithm continuously analyses response patterns and asks questions based on relevance and variation. This means that each user gets different questions every week. The system also uses sophisticated artificial intelligence to eliminate if an answer was a one-time answer or a trend. The ultimate goal for our AI is to find a balanced way to distribute all the available questions from each category (including supplemental and custom questions if they are active).
Flooding the AI with questions
If you are a new company joining Winningtemp, it can be tempting to send out all the questions simultaneously to get more data at the start. However, we highly recommend following your Onboarding Specialist's or Customer Success Manager's recommendations to give the system up to a month (usually three weeks) before you see data and temperatures.
By flooding the system with questions (sending out all questions at the start), it will affect the AI's ability to find balance. Ultimately, the AI algorithms become ineffective when you send out too many questions. You will have more data, but the data might not reflect reality.
An example
If Lisa answered negatively on a leadership question, she will most likely receive an additional leadership question in the next mailing. The aim is to find patterns in how users answer questions to avoid it being just a “one-time thing”. The AI also considers how many questions Lisa previously answered about leadership to ensure the mailings are varied.
When you flood the system with questions, the AI notices that Lisa has already received many leadership questions, and on the next mailing, she will get a question about autonomy instead. In this case, we will not know if Lisa answered that she had a bad Monday or if it's a negative trend. Depending on how many questions there are in the leadership category in relation to the other categories, it could take a long time before she receives new leadership questions. That's because the system tries to find a balance between the categories.
What about Insights?
Our AI generates Insights by analysing how users answer questions over time to find patterns. Flooding the system with questions could make the Insights irrelevant and inaccurate. The AI needs to be able to analyze how users answer over time, and by doing that, we can predict and give you valuable insights.
The solution
The solution is easy: follow the recommended survey length. The picture below recommends sending out six questions per week based on your mailing frequency and the number of active questions.
Activating more questions, such as the supplemental questions, and merging your mailings with the standard mailing will naturally raise the recommendations. This is normal, and it's because the AI has more questions to distribute. The difference here is that as long as we stay within (or close by) the recommended survey length, our AI will find a way to distribute the questions in a balanced way.
In other words, as long as you follow the survey length recommendations, you don't need to worry.