Our survey engine is very customizable, and it's easy to change the number of questions sent out in each mailing. Changing the mailing frequency is usually not a problem, but changing it to extreme values and sending out a lot more questions than we recommend comes with a price. Let's go through how the system works and what could happen if you send out too many questions.
How our AI works
Winningtemp's algorithm continuously analyses response patterns and asks questions based on relevance and variation. This means that each user gets different questions every week. The system also uses sophisticated artificial intelligence to eliminate if an answer was a one-time answer or a trend. The ultimate goal for our AI is to find a balanced way to distribute all the available questions from each category (including supplemental and custom questions if they are active).
Flooding the AI with questions
If you are a new company joining Winningtemp it can be tempting to send out all the questions at the same time to get more data in the start. However, we highly recommend that you follow your Onboarding Specialist's or Customer Success Manager's recommendations to give the system up to a month (usually three weeks) before you see data and temperatures.
By flooding the system with questions (sending out all questions at the start), it will affect the AI's ability to find balance. Ultimately, when you send out too many questions, the AI algorithms become ineffective. You will have more data, but the data might not reflect reality.
If Lisa answered negatively on a leadership question, she will most likely receive an additional leadership question in the next mailing. The aim here is to find patterns in how users answer questions to avoid that it was just a “one-time thing”. The AI also takes into consideration how many questions Lisa previously answered about leadership to make sure the mailings are varied.
When you flood the system with questions, the AI notices that Lisa already received lots of leadership questions and on the next mailing she will instead get a question about Autonomy. In this case, we will not know if Lisa's answer was that she had a bad Monday or if it's a negative trend. Depending on how many questions there are in the leadership category in relation to the other categories, it could take a long time before she receives new leadership questions. That's due to the fact that the system tries to find a balance between the categories.
What about Insights?
Our AI generates Insights by analysing how users answer questions over time to find patterns. By flooding the system with questions, the Insights could be irrelevant and inaccurate. The AI needs to be able to analyse how users answer over time, and by doing that we can predict and give you valuable insights.
The solution is easy, follow the recommended survey length. In the picture below, the recommendation is to send out six questions per week, based on your mailing frequency and the number of active questions.
If you activate more questions, such as the supplemental questions, and merge your own mailings with the standard mailing, it will naturally raise the recommendations. This is normal, and it's because the AI has more questions to distribute. The difference here is that as long as we stay within (or close by) the recommended survey length, our AI will find a way to distribute the questions in a balanced way.
In other words, as long as you follow the survey length recommendations, there's no need to worry.
2021-11-22 - Article created