This tutorial shows how to create a multi-wave project by cloning the project workspace from a previous wave of the same study. If you want to create a new project and set up the questions from scratch, check out:
Data considerations
Before creating a multi-wave project, make sure the new wave data is compatible with the prior wave data. This means:
- The same questions have the same question numbers
- Sunset any question numbers for deleted questions
- Use new question numbers for new questions
- There is a column that indicates "wave" (if not, we can also create this is Protobi's data process)
1) Create a new project
With each new wave of data, you can choose to create an independent project workspace. To get started, Create a new project using the current wave's data file (partial data works fine).
2) Copy the prior wave’s elements into the new workspace
The elements configuration determines the organization of the project (i.e. question order, titles, formats, etc...). Copying elements over is useful for tracking studies because the bulk of the survey stays the same or is very similar to the prior wave, so keeping the same organization saves a lot of time. Open the elements from project settings of the prior project and copy it over to the elements page of the newly created project.
See the duplicate a workspace tutorial for a GIF on how to copy elements from one workspace to another.
3) Bring prior wave processed data into the new project
For historical data from a prior project we prefer to use the processed data, otherwise we will have to be sure to apply the same processing code to the historical waves if you're starting from a raw aggregate wave data file.
In the data tab of the prior wave find the "Primary" data file. It could be a blue data table, but the primary data is typically a green data process which includes code where we applied business rules to clean your survey data based on your instructions.
Click into the "Primary" data table or process, and download as CSV. Save this file under a name like "process_w1" or "main_w1" etc.
4) Upload prior wave data as a table in the new project
In the new project space, the data file in "main" is the recent wave of data. Create a new data table for prior wave data.
Name the data table "process_w1" or something that represents the prior wave. Drop in the "Primary" file from the prior wave that you saved as a CSV.
5) Add a data process
After you add the prior wave's data to the new project space, create a new process by pressing on the "New process" button.
Press "Edit Run" to add code to this new process.
Bring prior process code into the new project
Copy the code from the process in the prior wave project over to the process you created in the new project.
Remember, we already added the prior wave's processed data, so there's no need to run the same process code on historical waves. You only need to carry over the code if you with to run it on the new wave of data.
Add code to stack prior wave with current wave
Add the code highlighted in yellow below that takes the data tables and creates a simple stack.
Simple stack code
/**
* @by tiffany@protobi.com
* @created 2023-06-01
* @see (enter URL of Freshdesk support ticket or other reference)
* This process assigns wave values and stacks the W1 processed data with the W2 data
* The W1 data table has the already processed W1 data from the W1 project workspace, the W2 table is the raw W2 data file
*/
var W1 = data["process_w1"] //define prior wave file
var W2 = data["main"] //define W2 (current wave) file
//TL, 2023-06-01, data from W1 is assigned wave = 1
W1.forEach(function(row){
row.wave = 1; //assign a wave value if it's not already programmed in the survey
})
//TL, 2023-06-01, data from W2 is assigned wave = 2
W2.forEach(function(row){
row.wave = 2; //assign a wave value if it's not already programmed in the survey
//TL, 2023-03-01,Code to create productx_enthusiast flag (Carry over each wave)
if (row.S10 >= 6 && row.S6_5 == 1) row.productx_enthusiast = 1
else row.productx_enthusiast = 0
})
var rows = Protobi.stack_rows([W1 ,W2]); //TL, 2023-06-01, stack prior and current waves of data
//TL, 2023-07-13
//Create new prescriber variable (Carry over each wave moving forward)
rows.forEach(function(row){
if (row.s1 == 3 && row.s4 > 0) row.prescriber = 1
else row.prescriber = 1
})
return rows; //return the resultHow to create a “wave” variable
If you don't already have a "wave" variable in your survey we can create one in Protobi. You will need this variable to filter on particular waves of data and to crosstab for wave comparisons.
In the same data process used to combine the multiple wave of data, we can define a wave variable. Create separate ".forEach" functions for each data table. Create a variable "row.wave" and assign it the appropriate value.
Wave assignment code
/**
* @by tiffany@protobi.com
* @created 2023-06-01
* @see (enter URL of Freshdesk support ticket or other reference)
* This process assigns wave values and stacks the W1 processed data with the W2 data
* The W1 data table has the already processed W1 data from the W1 project workspace, the W2 table is the raw W2 data file
*/
var W1 = data["process_w1"] //define prior wave file
var W2 = data["main"] //define W2 (current wave) file
//TL, 2023-06-01, data from W1 is assigned wave = 1
W1.forEach(function(row){
row.wave = 1; //assign a wave value if it's not already programmed in the survey
})
//TL, 2023-06-01, data from W2 is assigned wave = 2
W2.forEach(function(row){
row.wave = 2; //assign a wave value if it's not already programmed in the survey
//TL, 2023-03-01,Code to create productx_enthusiast flag (Carry over each wave)
if (row.S10 >= 6 && row.S6_5 == 1) row.productx_enthusiast = 1
else row.productx_enthusiast = 0
})
var rows = Protobi.stack_rows([W1 ,W2]); //TL, 2023-06-01, stack prior and current waves of data
//TL, 2023-07-13
//Create new prescriber variable (Carry over each wave moving forward)
rows.forEach(function(row){
if (row.s1 == 3 && row.s4 > 0) row.prescriber = 1
else row.prescriber = 1
})
return rows; //return the result
The result is an element that you can click to filter on, use to set global filters and use as a crosstab banner.
Set the new process as the Primary source of data
6) Edit the project to reflect survey changes
If your survey is exactly the same between waves, and you're using the same "elements" configuration as the prior wave then you shouldn't need to make any changes to the project space.
New questions are automatically added to fields
After you combine the current wave of data with prior waves, any new questions that were added to the survey will appear as new elements in "fields". There might be some variables that were already organized into the "fields" tab from the prior wave's elements setup, but new questions will be at the very end of "fields' list of children.
Organize new questions manually
Once you identify the questions that are new to this wave, you can create a new tab with a name like "new_wave2" and move the new questions there. Another option is to directly move the questions to their respective sections of the survey instead of consolidating them on one tab. You can optionally set the icon color on new questions to a color that is different than the standard color to easily differentiate the new questions from the old.
Manually moving questions like this works well for new waves where there are not many new questions. For waves where there are many new questions, you can choose to run autogroup instead.
Use autogroup to organize new questions
If the new wave of your survey added many new questions, you can choose to run autogroup on fields to avoid manually moving questions. Autogroup will group questions with similar keys together. For instance if there is a new option S13_5 with the S13 question, autogroup will put S13_5 under the existing S13 group.
Hide/remove questions
If you added/dropped questions between waves, then the new questions will need to be set up and the dropped questions should be moved or hidden from view.
7) Use global filters for each wave
For tracking studies, you can use global filters to see different waves of data individually. In the example below, we have a global filter on wave 2.
8) Update/refresh current wave data
Users usually create projects with partial data files and update the data as their surveys progress in fielding. To update the data in a tracking project upload the new data file into the "main" table (or whichever table contains the current wave)and re-run the process. For more information see the update project data tutorial.