The flipped model is feasible in universities, since students are used to modern technology (Sohrabi and Iraj, 2016; Subramanian and Kelly, 2019). They also have internet access on most university campuses and homes (Bergmann and Sams, 2012). It allows them to view the web-based instruction at their own pace. They can also watch the digital content as many times as they require. The model favors students who need more time to assimilate knowledge or have attention problems. Another element that promotes the employment of the model is that educators are interested in maximizing their students learning through innovative practices (Hao and Lee, 2016). Based on these elements, the flipped classroom model was used in several areas of knowledge such as Social Sciences, Technology, Engineering, etc.

In Engineering, the teaching-learning approach is represented by technical criteria, within a strong background in Mathematics and Physics (Halbe et al., 2015). Despite this particular background, the flipped model was employed in several Engineering courses with positive results. For example, in Fluid Mechanics, students increased their score in 7,5 points compared with the traditional model group (Webster et al., 2014). In the course of Mechanics of Materials, students in the flipped model performed better than the students in the traditional method (Lee et al., 2015). In an online course of Dynamics, the flipped model improved their learning experience and their problem-solving skills (Fredericks et al., 2013). In Classical Mechanical, most students preferred the flipped classroom model to the traditional one (Bates and Galloway, 2012).

In Civil Engineering, several experiences were conducted employing the flipped classroom model. For example, in the course of Water Resources, students were more involved, and improved their understanding of concepts and problem-solving skills (Li and Daher, 2017). In the course of Statics, learners improved their final grades, became more independent, and they were actively engaged with their learning (García-Ramírez, 2018). Students in the course of Mechanical Engineering, Computing, and Construction Materials admitted that the method had a positive impact on their learning (Gardner et al., 2014). In the course of Transport Engineering, students were satisfied with the flipped model, because they had the opportunity to work at their own pace, interact with their instructor individually, and work collaboratively with their classmates (Karabulut-Ilgu et al., 2016). Despite these positive experiences, in Civil Engineering it has not been employed in the Pavements design field, except for one preliminary study conducted by the author (García-Ramírez, 2019).

Notwithstanding the positive results in Engineering and Civil Engineering, some elements must be considered when applying the flipped classroom model. First, teachers should create quality videos with clear instructions. Second, students cannot ask questions to the teacher during the video (Milman, 2012). Third, students can be distracted by browsing other websites (Roehl et al., 2013). Fourth, in-class and out-class activities must be relevant to the student. Fifth, learners must have devices where they can watch videos. Finally, students should have access to the Internet. The first four elements offer some opportunities to improve the execution of the flipped classroom model.

Three groups participated in this study to answer both research questions. Group A took the course with the traditional model, while Group B took it with a flipped classroom, and Group C experienced it with a reinforced flipped model. Group A had 38 students, Group B had 57 students, and Group C had 70 students. They enrolled in the course in the academic periods of April-August 2017, April-August 2018, and April-August 2019, respectively. All groups were in the fourth year (out of 5) of the Civil Engineering program at the Universidad Técnica Particular de Loja (UTPL). In this course, students learnt how to perform the pavement structural design for highways and airports. The teacher was the same in all groups. Before this course, students had to approve the courses: Highway Geometric Design, Materials, Soils and Rocks, and Physics.

The learning models used for the three groups are shown in Table 1. In the traditional model, students had to study some material shared through the Virtual Learning Environment (VLE) or in books. Then, they attended the face-to-face class, where they sometimes had to solve reading controls at the beginning of the session. Later, they received a lecture about the week topic. Finally, they participated in a problem-solving session individually o in teams, using a solved example from books. The teacher showed some essential parts to solve it and in the end, the answer to the problem. The teacher helped if they had questions to ask. After in-class activities, students had to solve other problems as homework. They could attend a weekly tutoring session if they had questions when solving or trying to solve the homework.

Table 1

Group	Before class	Face to face class	After class
A	Study some material	Solve reading controls Receive a lecture Problem-solving session	Solve problems as homework Go tutoring
B	Watch a pre-recorded lecture on YouTube	The instructor solve questions about the topic Problem-solving session	Solve problems as homework Go tutoring
C	Watch a pre-recorded lecture on Edpuzzle	Solve reading controls The instructor solve questions about the topic or the field visit Problem-solving session	Solve problems as homework Go field visits to some building project Go tutoring

Source: Authors

In the flipped model, students have to watch a pre-recorded lecture (in a video) before attending the class. The videos were uploaded on the You tube platform and shared through the VLE. To check if they watched them, learners had to solve reading controls at the beginning of the class. Then, the instructor answered questions about the weekly topic and explained their main concept. Afterward, students participated in a problem-solving session individually o in teams, using a solved example made by the teacher, who also showed some essential parts to solve it and in the end, the answer to the problem. The teacher helped them if they had questions about it. They also had to solve another problem as homework. A weekly tutoring session was available if students had questions about the problem and /or the weekly topic.

In the reinforced flipped classroom model, students had to do the same as in the flipped model, but including field visits to a building project related to the course contents. These visits are conducted by the students on their own and without the presence of the teacher. After the visit, they had to make a report about the activities in the construction work. For every activity, photos were mandatory. These photos should include the student, as well as the date and time, to avoid sharing between them or reusing the same photos in their next reports. In this model, the teacher was more involved in their learning, giving feedback in every academic activity, and explaining its aim. So, students knew their mistakes and understood the reason for each activity. Considering that students in group B could skip watching the videos on You tube, in this case the videos were uploaded on the Edpuzzle platform. In this platform, after students created their users, they had to watch the previously scheduled pre-recorded lecture (start and due date). In this platform, some questions could be added during the video (multiple selections, true/false and short answer) and could set an option to prevent skipping, so students had to watch the whole video. The face-to-face class started with the control reading, and students asked about the field visits or the weekly topic.

All three groups had the same grading procedure and grading factors. Reading controls had a weight factor of 20% of the final score grade. The in-class problem-solving session was 15% of the final score, as well as for solving problems as homework. This homework could include up to two more problems related to the week class. The remaining 50% were the midterm exams. Students could get a maximum score of 100 points, and they should have a minimum of 70 points to approve the course.

The course was 16 weeks long. All three groups had the same course content, as shown in Table 2. This table also included the duration of the pre-recorded lectures. In some weeks, they had no video to watch, because it was the beginning or end of the course, or the midterm exams. Pre-recorded lectures were short according to the literature suggestions (Bonwell, C., and Eison, 1991; Enfield, 2013; García-Ramírez, 2018; Sohrabi and Iraj, 2016). Videos were made using digital slide presentations with voiceovers in Microsoft PowerPoint software.

Table 2

Week	Topic	Duration mm:ss
1	Introduction to Pavements	-
2	Traffic study	32:01
3	Soil study for the design of Pavements	26:10
4	Soil stabilization	29:08
5	Flexible pavement structural design for highways - Asphalt Institute method	21:47
6	AASHTO flexible pavement structural design for highways	26:23
7	SHELL flexible pavement structural design for highways	19:47
8 Midterm exam -
9	PCA rigid pavement structural design for highways	37:04
10	AASHTO rigid pavement structural design for highways	18:00
11	Articulated pavement structural design for highways method	43:13
12	FAA flexible pavement structural design for airports	24:19
13	PCA and FAA rigid pavement structural design for airports	16:08
14	Failures in Pavements	39:36
15	Maintenance and Rehabilitation - Pavement	-
16	Final exam	-
- There was no pre-recorded lecture to watch.

Source: Authors

Data collection was performed using two instruments: final score grades and a blind survey at the end of every course. The teacher calculated the student's final scores according to the grading plan. The average grade was estimated in every group, in order to compare it with other groups. A frequency distribution was also made with the individual final score grades to compare the differences between groups. The survey was optional, so 58 students answered it in group A, 55 in group B, and 66 in group C. The questionnaire was composed mostly of several closed-ended questions and a few open questions. Questions were about the course and the applied method.

The survey had three types of questions: numeric (rates), closed-ended, and open. It calculated the average rate from all answers of the numeric questions. In closed-ended questions, it estimated the percentage where students agreed (strongly agree and agree). This information provided an overview of the opinion of the students, since the other results (neither agree nor disagree, disagree, strongly disagree) are their correlative. Finally, in open questions, similar answers were clustered in several categories. When a response had no match with others, it was classified and grouped in the category "Others".

The final average score grades were: 72,82 (group A), 69,68 (group B), and 71,58 (group C). In order to find a statistically significant difference between those values, an analysis of variance (ANOVA) was performed using the statistical software Minitab 14.2 (2005). A 95% level of confidence was used as a parameter of this analysis. Scores from group A did not differ significantly from group B (p-value = 0,066), which means that traditional model had similar scores to the flipped model. Scores from group A also did not differ from scores from group C (p-value = 0,380), which means the flipped method did not increase the grades significantly when compared to the traditional one.

The individual final scores were drawn in a histogram to analyze their distribution in all study groups (Figure 1). Group A had 68% of students with equal or more than 70%, while group B had 49%, and group C had 51%. Most students had between 70 and 79 points in all groups. Students (2%) with the highest score belong to group B. In the range of 80-89 points, there were 13%, 4%, and 30% of students, from group A, B, and C, respectively. Group A had 19% of learners that obtained between 60 and 69 points, that is, close to passing the course. However, it is interesting that the flipped models had more percentage in this range, 33% (group B) and 30% (group C). Even though, more students approved the course under the traditional model, the flipped models had more distributed scores. In addition, they had some interesting trends and behaviors in some ranges that traditional models did not have.

Figure 1 Source: Authors

In order to see if groups were academically homogeneous, the difference between the grades was analyzed in the first part of the semester (up to the midterm exam) and compared to the second one (up to the final exam). Most students in group A (76,47%) got the same or a higher grade in the second midterm than in the first one. Students from group B performed a little lower (63,16%), and group C was the lowest (21,74%). These differences could mean that students in group A are better students, in terms of grades, than those in other groups. This could also mean that learners in group A were more motivated than other students in group B and C. These assumptions should be analyzed in future research.

The survey had several parts: rating, selecting options, and responding to open questions. In the rating section, students were asked to rate the course and their self-learning throughout the course. Table 3 shows their average rating values for both questions. The traditional model got a higher average rate than the flipped classroom, and this one got a smaller value compared to the reinforced flipped classroom model. ANOVA analysis was conducted to see if there was a statistical difference between groups of answers. The average rate between group A-B and A-C did not differ statistically (p-value = 0,318 and p-value = 0,126, respectively). However, the average rate between group B-C differs (p-value =0,003), which suggests that the students' opinion about the method is more positive in the reinforced flipped classroom than in the simple flipped model.

Table 3

Source: Authors

As expected, concerning self-learning, both flipped classroom models got higher average rates than the traditional one. In addition, an ANOVA analysis was performed with those answers. An average rate differs statistically between groups A-B (p-value = 0,037), but not in groups B-C (p-value = 0,367). This implies that in the flipped models, students are more aware of their self-learning than in the traditional one and, consequently, they consider that they work more than the students of group A.

Students also responded about the methods executed with closed-ended questions. These items had a five-point Likert scale: strongly agree, agree, neither agree nor disagree, disagree, strongly disagree. Table 4 included the percentages of the agreement answers (agree and strongly agree).

Table 4

N° of question	Percentages	of students that agreed (strongly agree and agree)
	Group A	Group B	Group C
3	98,28	98,18	100,00
4	93,10	90,91	95,45
5	93,10	90,91	96,97
6	87,93	87,27	90,91
7	96,55	96,36	100,00
8	87,93	72,73	95,45
9	79,31	69,09	90,91
10	82,76	72,73	87,88
11	-		93,94
12	-		86,36

3 The topics of the subject were interesting.

In general terms, topics were interesting for all groups. Most students estimated that in-class and out-class academic activities were good enough and in accordance with their expectations for all three groups. Additionally, most of them thought that the teacher mastered the topics of the subject. Grading had a lower score in the flipped model than in the others, which means that learners were unsatisfied with the grading scores. Interestingly, the reinforced flipped model got the highest score when students answered about the implementation of the method. However, the flipped model had the lowest score, which means that the students evaluate other aspects, independently of the learning method.

The reinforced flipped group has the highest percentage between groups in all of these questions. Besides, when they perceived that the method was well implemented, they suggested that it should be used in other courses of Civil Engineering. Most students under the reinforced flipped model admitted that the visit fields helped in their learning and complemented the other academic activities. Most of them also considered that the reinforced flipped model was better than the traditional one.

Regarding the few open questions that students answered, one question focused on what changes they suggest for the next course of Pavements. Table 5 shows their answers to those questions after processing them. It is remarkable that they suggested fewer modifications in the flipped models than in the traditional one. This means that in the flipped models, students are more satisfied than in the traditional one.

Table 5

A: No modifications, B: complements with field visits to the pavement Works in the highways around, C: the teacher should solve and explain the problems first, before students solve their problem example, D: teach asphalt mixtures theory or the course should have laboratory practices, and E: others. No student suggested this item.

Some students from the traditional model and few from the flipped model suggested that field visits should be executed to learn more. For that reason, in the reinforced flipped model, field visits were implemented. In the traditional model, some students required that, at the beginning of the class, the teacher should solve and explain a problem, before students solve their problem example. Therefote, in group B, instead of solving the homework problem, the teacher solved a different problem as guiding document, with similar input parameters to their problem example. However, in group B, few students made the same request again. So, in group C, in addition to the guiding document, feedback was implemented, and very few students complained about it. The teacher did not want to show them how to solve the problem because he/she considered that students could learn more and more permanently if they solved it on their own. Furthermore, some students in group A and C asked for asphalt mixtures theory or including laboratory practices as part of the course teaching plan. Maybe, they made this requirement because previous teachers included those topics. Finally, some students had other requests that ended up in the "other" category.

Students in the flipped classroom models also answered if the videos helped them in their learning process. Most students (98,18%) in group B and group C (98,48%) said that pre-recorded lectures helped them in the course. These answers were clustered in 6 categories, as shown in Table 6.

Table 6

F: they helped to understand additional information and clear doubts, G: they can be rewatched or paused, H: they helped to know in advance the topic of the next class, I: with them, it is easier to understand than in books, J: they helped learning additional concepts that are used in real life, and K: others.

Two additional open questions were asked in group C in relation to the feedback process and field visits. Most students (83,82%) recognized that they agreed with the feedback given by the teacher. Some of them (10,29%) considered that feedback should include more detailed observations or include videos, documents, or the solved problem. A few students (5,88%) asked for clearer feedback. Concerning the field visits, most students (39,17%) agreed with the frequency and report that they should be done. Other students (16,18%) considered that the teacher should be with them and solve their doubts in-situ. Some students (11,76%) answered that field visits should be arranged in more significant building works. Few students (8,82%) demanded that selfies should not be included in the reports, and others (16,18%) had minor requests that were not categorized.