Welcome to the SynergyLMM tutorial!
This quick-start tutorial shows how to use the web-app. An example dataset will be used, which can be downloaded by clicking in the link below:

Download Example Dataset

The user can upload the data in the 'Model Estimation' tab. The input data should be a data table in long format with at least four columns: one for sample IDs, one for time points, one for treatment groups, and one for tumor volume measurements. (See the example dataset for reference).

This is how the data should look like:

Once the data is uploaded, the user will need to specify which columns contain each information. For this example, columns Mouse, Day, Group, and TumVol contain the information about the sample IDs, time points, treatments, and tumor measurements, respectively (Fig. 1a).

The next step is to define the time points for the experiment. By default, the minimum and maximum values in the 'Time' column will be used as the start and end time points, but these can be modified. The name of the different treatments in the experiment also need to be specified. In this example, 'Control', 'DrugA', 'DrugB', and 'Combination' correspond to the control, single drugs, and the drug combination groups (Fig. 1b).

Next, clicking on 'Run Analysis' will fit the model and produce a table with the model estimates, (Fig. 1c), including the tumor growth rates of the different groups, and the standard deviations for the random effects and for the residuals. If 'Show Plot' is selected, the plots with the tumor growth curves are displayed along with the regression line for the fixed effect coefficients (i.e., the tumor growth rates for the control and treatment groups, Fig. 1d).

The orange-shaded lines in Fig. 1d indicate two individuals, mouse 2 and mouse 10, whose measurements are (intentionally) notably different from the others, and which will greatly affect the model and the results. The impact of these individuals and how to deal with them will be discussed later.

Minimum Observations per Subject and Growth Model

In Fig. 1b, there is an option called 'Minimum Observations per Subject' , which allows for excluding samples with fewer than the specified number of observations. For example, in Figure 1d, the arrow heads indicate subjects for which only four observations were made. If the the minimum observations per subject is set to five, these samples will be omitted from the analysis (Fig 2).

Fig. 1b also shows an option called 'Growth Model' . This box allows to choose between 'Exponential' or 'Gompertz' growth models. The exponential model is fitted using a linear mixed effect model, while the Gompertz model is fitted using a non-linear mixed effect model. The exponential model is generally a good starting point for most in vivo studies, and it tends to converge more easily, it may be insufficient when tumor growth dynamics deviate significantly from exponential behavior. In such cases, the Gompertz model typically offers better flexibility and more accurate estimates of tumor growth and treatment effects.

Once the model is fitted, the Synergy Analysis can be performed. Fig. 3a shows the different options for synergy analysis. The user can choose between Bliss, highest single agent (HSA), or response additivity (RA) reference models. In the advance options, selecting 'Use Robust Estimators' will apply sandwich-based cluster-robust variance estimators. This is recommended and can help to deal with possible model misspecifications.

The analysis for the response additivity reference model is based on simulations, and the number of simulations to run can also be specified by the user.

The output of the analysis includes a plot with the combination index (CI) and synergy score (SS) for each time point (Fig. 3b), as well as a table with all the results. The plot in Fig. 3b shows the estimated synergy metric (CI or SS) value, together with the confidence interval (vertical lines), and the color of the dots indicates the statistical significance.

The interpretation of the CI and SS values follows the typical definition of CI and SS:

• CI: Provides information about the observed drug combination effect versus the expected additive effect according to the reference synergy model. A drug combination effect larger than the expected (CI > 1) indicates synergism, a drug combination effect equal to the expected (CI =1) indicates additivity, and a lower drug combination effect than the expected (CI < 1) indicates antagonism. The value of the CI represents the proportion of tumor cell survival in the drug combination group compared to the expected tumor cell survival according to the reference model.
• SS: The synergy score is defined as the excess response due to drug interaction compared to the reference model. Following this definition, a SS>0, SS=0, and SS<0, represent synergistic, additive and antagonistic effects, respectively.

As observed in Fig. 3b and the table with the results, the only time point for which there is a statistically significant result is time 3. At this time point, the CI is lower than 1, and the SS is higher than 0, indicating synergy in the drug combination group. However, at the other time points, the results show that the confidence interval includes 1 for the CI and 0 for the SS, and the p-value is higher than 0.05, indicating that the additive effect of the drug combination cannot be rejected.

In addition to the synergy results, a table output with the model estimates is also returned. In the case of using the exponential model, as a model is fitted for each time window being analyzed, the model estimates for each time point are reported:

An important step in the analysis is to verify that the model is appropriate for the input data. In the Model Diagnostics tab, the user can check the main assumptions: the normality of the random effects and the residuals.

The model diagnostics output includes four plots to help evaluate the adequacy of the model. The two plots at the top of Fig. 4a show the Q-Q plot of the random effects and normalized residuals. Most points in these plots should fall along the diagonal line. The two plots at the bottom of Fig. 4a show scatter plots of the standardized residuals versus fitted values and standardized residuals per time and per treatment, which give information about variability of the residuals and possible outlier observations. The residuals in these plots should appear randomly distributed around the horizontal line, indicating homoscedasticity. Fig. 4b shows the output of Shapiro-Wilk normality tests for the random effects and the residuals. A non-significant p-value obtained by the Shapiro-Wilk normality indicates that there is no evidence to reject normalilty.

The results from the plots and the normality tests seem to indicate some departure from the normality, especially evident for the residuals. This could be related to those individuals which has a large variation in the measurements.

Observations with absolute normalilzed residuals greater than the 1−0.05/2 quantile of the standard normal distribution are identified as potential outlier observations. A table with these observations is also provided by SynergyLMM. It can be observed that many observations from mouse 2 and mouse 10 have large residual values.

As mentioned before, mouse 2 and mouse 10 are intentionally introduced outliers in the model, and there are many observations from these subjectes that are indeed identified as potential outliers. More details about outlier identification are provided in the Influential Diagnostics section. But first, there are several approaches that the user can follow to try to improve the model diagnostics.

If the diagnostic plots and tests show evident violations of the model assumptions, there are several solutions that may help improving the model:

- Transform the time variable: Both exponential and Gompertz models are based on ordinary differential equations (ODEs), which can be formulated using arbitrary time scales. Applying transformations, such as the square root or logarithm (with +1 offset), can often improve model fit, enhance linearity, stabilize variance, and ensure better numerical performance. Importantly, when comparing the exact values of a drug combination synergy index, the models should use the same timescale

- Specify a residual variance structure: SynergyLMM uses the ‘nlme’ R package, which supports flexible variance and correlation structures. These allow for heterogeneous variances or within-group correlations, such as those based on subject, time point, treatment, or combinations of these. Applying these structures can substantially improve model diagnostics and the robustness of the estimates. This can be done in the Advanced Options of the Model Estimation tab.

- Prefer the Gompertz model when the exponential model fails: While the exponential model is generally a good starting point for most in vivo studies, and it tends to converge more easily, it may be insufficient when tumor growth dynamics deviate significantly from exponential behavior. In such cases, the Gompertz model typically offers better flexibility and more accurate estimates of tumor growth and treatment effects.

- Carefully address potential outlier: Individuals or measurements highlited as potential outliers may warrant further investigation to reveal the reasons behind unusual growth behaviours, and potentially exclude these before re-analysis, after careful reporting and justification.

In this case, the model will be fitted especifying a unequal variance for the errors. Concretely, a different variance for each treatment is defined (Fig. 5a). It can be seen that this has improved the model diagnostics, and now both the random effects and normalized residuals are approximately normally distributed. There are still some potential outlier observations, which is normal, since there are two intentional outlier subjects.

Another important tool for the diagnostics of the model is the plot of the observed vs. predicted values, available in the Model Performance tab. The output from this tab includes a table with model performance metrics, such as R-square and various error measures (Fig. 7a), along with the plot of observed vs. predicted values (Fig. 7b).

Each plot in Fig. 7b corresponds to a subject. The blue dots represent the actual measurements, the continuous line shows the group (treatment) marginal predicted values, and the dashed line indicates each individual's predicted values

By examining how closely the lines fit the data points, the user can assess how well the model fits the data. In this example, the model generally fits the data well, except for certain subjects, such as mouse 2 and mouse 10.

Power Analysis

Note: Power analyses are only available for models fitted using the exponential growth model.