To achieve the results, 180 students who were standing in the same level of language knowledge-an intermediate level of English proficiency- in their mean score (16.53 ± 0.35.), including male and female students, speaking Farsi as their first language, with the average age of 23 years old, were randomly assigned to the two experimental and control groups, in which E-learning resources and traditional references trainings were provided as the treatment of the study via the translation assignments on the translation quality of the learners. To obtain the expected results, a two-group pretest-posttest experimental design was used, the results of which were summarized as follows.
4.1. Data Analysis
4.1.1. Inter-Rater Reliability Indices
To ensure the existence of reliability between the raters’ scores in both pre-test and posttest results, as shown in
Table 1, Cronbach’s alpha was calculated. The results indicated the existence of a high consistency between the two translation raters in their scoring systems since α was 0.933; which was greater than > 0.70.
Reliability Statistics via Inter-Item Correlation Matrix
Inter-Item Correlation Matrix R1_Pretest R2_Pretest R1_Posttest R2_Posttest R1_Pretest Scores 1.000 0.957 0.724 0.713 R2_Pretest Scores 0.957 1.000 0.731 0.731 R1_Posttest Scores 0.724 0.731 1.000 0.923 R2_Posttest Scores 0.713 0.731 0.923 1.000 Intraclass Correlation Coefficient Intraclass Correlation Confidence Interval (95%) F Test with True Value 0 Lower Bound Upper Bound Value df1 df2 Sig Single Measures 0.778 0.714 0.833 14.982 99 297 0.001 Average Measures 0.933 0.909 0.952 14.982 99 297 0.001
aCronbach’s alpha: 0.933; standardized items Cronbach’s alpha: 0.940; number of items: 4
Table 1, both pretest and posttest results from the two raters showed correlation with each other. Besides, there existed a high degree of interrater reliability between the two raters at both pre and a posttest results, since the results for intraclass correlation for average measures was 0.933.
Besides, the means of the two groups at both pretest and posttest results are shown in
Pretest and Posttest Results in the Traditional References and E-learning Resources Training Groups
N Minimum Maximum Mean Std. Deviation Traditional references training Mean_pretest 45 8.00 14.50 11.25 2.86 Mean_posttest 45 11.50 16.00 13.75 2.02 Valid N (listwise) 45 E-learning resources training Mean_pretest 45 10.00 16.50 13.96 2.22 Mean_posttest 45 13.00 19.00 15.51 1.75 Valid N (listwise) 45
The descriptive statistics provided in
Table 2 represent how different groups varied through the means gained at both the traditional and E-learning resources pretest and posttest groups of training of translation. In other words, the translation training via E-learning resources made a statistically significant difference on the learning outcome of the students and the quality of students’ translation via obtaining the mean of 15.51, which was greater than the mean of the participants in the traditional references group (13.75). 4.1.2. Analysis of Covariance-ANCOVA
ANCOVA was used since the researcher aimed at controlling the potential effect of covariate, the pretest of translation, via considering the preexisting difference between the experimental and control groups and its potential effect on the dependent variable of the study, the posttest of translation. The results of ANCOVA test are shown in the following tables by identifying the two levels of the between-subjects effects.
As shown in
Table 3 the P value of 0.415 is greater than 0.05 and it represents the fact that the interaction between the covariate (pretest) and independent (group) variable is not statistically significant for training translation via applying E-learning resources and traditional reference works. Thus, the researcher had not violated the assumption of homogeneity of regressions which was revealed through the similarities of the groups with respect to their slopes and trends. In other words, the factors (pretests) and covariates (experimental and control groups) did not interact.
Pretest and Independent Variable Interaction via Between-Subjects Effects Results
Source Dependent Variable: Translation Posttest Sum of Squares (Type III) df Mean Square F Sig. Group 0.266 1 0.266 0.173 0.679 Pretest 172.269 1 172.269 111.885 0.001 Group* Pretest 1.034 1 1.034 0.671 0.415 Error 147.811 96 1.540 Total 25351.875 100 Corrected total 356.265 99
To complete the quantitative analysis of the study, the test of the main hypothesis was carried out through the tests of between-subjects effects, the results of which are presented in
Table 3. The table shows whether our groups in the study were significantly different in terms of the scores on the outcome, which was the posttest score based on the effect of independent variable through the test of between subjects effects. In order to interpret the results- “P value”, shows the statistical significance of whether there are statistically significant differences in posttest scores (i.e., the dependent variable) between the groups (i.e., the independent variable) when adjusted for pretest scores (i.e., the covariate). In this respect, it is possible to consider whether there is a statistically significant difference between adjusted means (P < 0.050).
As shown in
Table 4, training translation via applying E-learning resources’ main effect was significant on the students’ translation proficiency development [P (0.001) < (0.050)] controlling for the effect of pretest and the null hypothesis was rejected. In other words, One-way ANCOVA was conducted to determine the difference between the pretest and posttest scores on translation proficiency of students controlling for the potential effect of pretest scores of participants by experimental and control groups, through which P = 0.001 < 0.050 resulted in rejecting the null hypothesis.
Test of the Main Hypothesis via Between-Subjects Effects Results
Source Dependent Variable: Translation Posttest Type III Sum of Squares Df Mean Square F Sig. Partial Eta Squared Pretest 200.130 1 200.130 130.422 0.001 0.573 Group 6.919 1 6.919 4.509 0.001 0.244 Error 148.845 97 1.534 Total 25351.875 100 Corrected Total 356.265 99
aR squared = 0.582 (adjusted R squared = 0.574)
Table 4, the significant value of 0.001 which is smaller than 0.050 indicated the fact that our groups were significantly different from one another. Also the effect size of 0.244 revealed the strength of the independent variable as an appropriate numerical value for indicating the efficacy of the treatment-training translation via applying E-learning resources.
Since the adjusted means of the groups based upon the influence of covariate of training translation via E-learning resources made the researcher observe the statistical significance of 0.001, the researcher could figure out which groups significantly differed from the others through the results of the post hoc test results shown in
Estimated Marginal Means the Experimental and Control Groups
Group Dependent Variable: Translation Posttest Mean Std. Error 95% Confidence Interval Lower Bound Upper Bound Control 15.547 0.175 15.199 15.895 Experimental 16.073 0.175 15.725 16.421
Further to this, the researcher could compare the outcomes from experimental and control groups and it was concluded that via applying E-learning resources in translation training, the groups were significantly different from each other through multiple measurements of P value, since P = 0.001 and is smaller than 0.050- representing how different groups vary. Accordingly, the translation training via E-learning resources made a statistically significant difference on the learning outcome of the students and the quality of students’ translation, while controlling the effect of the pretest.