THE NEW AND THE OLD FOR ENGAGEMENT IN BUSINESS LEARNING: GAMIFIED COURSE DESIGN AND OP-ED WRITING PROJECT

Achieving higher level of student engagement in learning is a popular topic among educators of various disciplines. Numerous methods have been explored to the end of making learning more engaging and students more motivated and yet, due to the everchanging nature of the “new generation”, there always seems to be something new to try. In particular, college students that major in business strive for opportunities to engage in hands-on activities that are relevant in the real-world to be better prepared for their professional careers after graduation. The advancements in computer technology and the rising popularity of videogames as entertainment media consumption gave rise to the study of how the design principles of games can be applied in other activities, such as learning, training, and management. Gamification is widely adopted in the industry for onboarding, training, performance management, and customer engagement. In the academia, gamification studies have grown in higher education in various disciplines such as business, medicine, and engineering. Recent gamification studies grounded in selfdetermination theory (SDT) call for proper gamified design to cast a deeper look into human behavior on what gives us a click in the mind to become engaged and motivated in activities typically viewed as distant from fun and exciting. The first manuscript, a study in gamified Operations & Supply Chain Management course design, explores what gamification does to student engagement, satisfaction, and content knowledge when applied restrictively to assessment activities within the course. The negative findings suggest that a wider, more comprehensive application is needed for the gamified course to be engaging, as well as lead to improved academic performance. While exploring what the new technique of gamification can offer to student engagement, it is worth exploring a legacy media with the same goal: newspapers. The popularity of television and internet forced the newspaper industry to adopt online publication as their business model due to diminished demand for print circulation. However, print and online local and regional newspapers still dominate the US market, making them a viable channel of engagement for students. The second manuscript presents a team project designed to guide students in applying their learning in an opinionated writing and submitting their voice to local and regional newspapers. Designed according to principles of active learning theory and team-based learning theory, the op-ed writing project presents a unique learning opportunity to students by taking what they’ve learned in the classroom and scaling the abstract constructs and principles down to their own community contexts in the neighborhood, local, regional, or national level, while interacting with teammates for a consensus opinion of their own. The op-ed writing project led to seven student op-eds published in local and regional newspapers. Survey results show positive student perceptions in realizing their opinion is worth sharing with the public, in learning a new style of writing, and in that the project helped them learn the course materials better, indicating a successful


LIST OF TABLES
As gamification becomes more popular due to its benefits on engagement and motivation, application in higher education has been focused on developing electronic learning management systems, similar to those used in the industry for employee management. However, heavy reliance on technology for superficial gamification is hindering wider adoption and exploration of gamified course design by individual instructors. Also, the effects of gamification when applied selectively to specific areas of a course has not been extensively studied.
We gamified an operations and supply chain management course by implementing game mechanics and elements selectively to assessment activities. We used surveys to measure the effects of gamification on student perceptions of satisfaction, course experience, learning, impact of teaching method. Student's content knowledge was measured by quizzes and exam scores.
Interestingly, we found that gamifying assessment activities resulted in significantly lower content knowledge, satisfaction, and course experience. Difference in perceived learning was not significant. Also, team exam scores were significantly lower in the gamified group, while individual exam scores were not significantly different.
This study provides empirical evidence that gamification in classroom may produce unintended consequences and implementing gamification restrictively to assessment is ineffective at best. A holistic approach for gamified design is needed for a gamified course to be successful in engaging students and to achieve higher academic performance Keywords: pedagogical research, teaching supply chain management, assessment, gamification.

INTRODUCTION
"Games are showing us exactly what we want out of life: more satisfying work, better hope of success, stronger social connectivity, and the chance to be a part of something bigger than ourselves." -Jane McGonigal, Reality is broken (2011, p. 114) In recent years, gamification has become more popular and pervasive in various contexts as a means of embracing the benefits of positive human emotions invoked by games (Seaborn & Fels, 2015;Dias, 2017;Subhash & Cudney, 2018). Gamification is noticed by academics, educators, and practitioners from a variety of domains, following a trend within the business and marketing sectors (Seaborn & Fels, 2015). Specifically, gamification has been utilized and found to enhance motivation and improve user experience in brand loyalty (Zichermann & Cunningham, 2011), healthcare and health awareness (Hamari & Koivisto, 2015;González et al., 2016), management and training (Saunders, 2017), and education and learning (Buckley & Doyle, 2014;Stansbury & Earnest, 2016). Gamification by itself has become an emerging segment in the industry, expected to grow to over US$ 22.9 billion by 2022 (P&S Market Research, March 2016).
With a goal to bring some of these benefits to our classrooms, we gamified the assessment activities of an undergraduate level operations & supply chain management (OSCM) course. The course is mandatory to all students majoring in business, and contains various concepts, constructs, and analytical content that requires high retention of student engagement for success. Specifically, reading the textbook to understand basic concepts and theories before lectures is crucial for student success, as well as to the efficient progression of the course. To this end, we implemented game mechanics and elements selectively to assessment activities of two sections of the OSCM course and measured the effects of gamification on student perceptions of satisfaction, course experience, learning, impact of teaching method, and student's content knowledge.
Contrary to our expectations, gamification of assessment activities hurt students' content knowledge, satisfaction and course experience. This study contributes to the literature by demonstrating a dark side of gamification in a classroom setting and by exploring the effects of selectively gamifying a course design element (assessment) with no online platform involved.
The rest of the paper is structured as follows. First, an overview of gamification and its application in higher education is provided, followed by the theoretical background for gamification. The basis for selecting assessment activities as the focal area for gamification is clarified, accompanied by the experimental design and details on gamified assessment in the course. The method section outlines the data collection protocol and survey items. The findings are reported in the results section, followed by detailed analysis in the discussion section. Finally, major findings and implications are outlined in the conclusion, followed by limitations and directions for future research.

Gamification in Higher Education
Gamification is emerging as a research topic in higher education among various disciplines (Seaborn & Fels, 2015). We adopt the definition of gamification as the "process of enhancing a service with affordances for gameful experiences in order to support user's overall value creation (Huotari & Hamari, 2012, p. 19)". Gamification differs from game-based learning (GBL) in that gamification uses elements and mechanics of videogames to improve user experience and engagement in non-game contexts (Aldemir et al., 2018), whereas GBL utilizes full-fledged games or videogames, often labeled "serious games", incorporated to the curriculum activities, as described by Kong (2019).
Subhash & Cudney, through a systematic review of the literature focusing on gamified learning in higher education, reveal a growing number of research in recent years, as well as put business-related research in the second most published subject area after computing (Subhash & Cudney, 2018).
A large body of gamification research has focused heavily on technology to create gamified experiences (Stansbury & Earnest, 2016). In the education literature, the majority of gamification research in higher education entail the use and/or development of a dedicated online platform or a gamified learning management system (G-LMS) (Villagrasa, Fonseca, Redondo, & Duran, 2014;de-Marcos, García-Cabot, & García-López, 2017;Dias, 2017). This focus and reliance on technology may be hindering the wider adoption of gamification in higher education, due to the difficulty of designing and managing a complex information system (Sobocinski, 2017). There is a paucity of research in how gamification can be implemented without using or developing a complex technological system. Also, research on gamification of specific course design elements is desired, but remains scarce (Nacke & Deterding, 2017) since most are focused on gamifying the full-scale course design and information systems development, which can be an overwhelming task and burden for any single instructor exploring the benefits of gamification without organizational support (Sobocinski, 2017).
Two recent exceptions stand out from these trends of technology dependence and full-scale points, badges, and leaderboards (PBL) approach in gamification research. Song (2017) designed a smaller-scale implementation of gamification focusing on the element of asking questions, examining the engaging effects of gamification within the individual and social interactions of the classroom (Song, 2017). Morillas Barrio, Munoz-Organero, and Sanchez Soriano (2016) found positive effects of gamified student response system (SRS) on student motivation, attention, and learning performance (Morillas Barrio et al., 2016). Their work also presented a novel path of gamification in higher education by implementing gamification around another innovative technique to enhance its benefits.

Theoretical Background
The majority of gamification research in the educational environment identify student engagement and motivation as core behavioral benefits of employing gamification (Hew, Huang, Chu, & Chiu, 2016;Kuo & Chuang, 2016;Subhash & Cudney, 2018;Van Roy & Zaman, 2018). Popularly cited in this stream of research is selfdetermination theory (SDT), which argues that "an understanding of human motivation requires a consideration of innate psychological needs for competence, autonomy, and relatedness (Deci & Ryan, 2000, p. 227)". Ryan and Deci (2000) identify the needs for competence, relatedness, and autonomy as essential for facilitating optimal functioning of the natural propensities for growth and integration, as well as for constructive social development and personal well-being . Deci and Ryan (2000) present a self-determination continuum, projecting type of motivation, type of regulation, and perceived locus of causality affecting varying degrees of selfdetermination. Intrinsic motivation is projected as most desirable type of motivation, in the extreme of self-determined behavior. Deci and Ryan (2000) also present intrinsic motivation, autonomous regulation of extrinsic motivation, and intrinsic aspirations as antecedents of high-quality performance, healthy behavior, and positive experiences.
Gamification research grounded in SDT can be categorized into structural and content gamification, according to the type of motivation (Kapp, Blair, & Mesch, 2014;Hudiburg, 2016). Structural gamification relies on external motivation by adding game elements (e.g. points, badges, leaderboards) without altering content, whereas content gamification utilizes intrinsic motivation by applying game elements, mechanics, and thinking to the content, making an activity more game-like (Kapp et al., 2014;Hudiburg, 2016).
In the educational gamification research, intrinsic motivation is widely cited from SDT as desired outcome (Hew, Huang, Chu, & Chiu, 2016;Kuo & Chuang, 2016;Subhash & Cudney, 2018;Van Roy & Zaman, 2018). Intrinsically motivated students are more engaged, retain information better, and are generally happier (Hanus & Fox, 2015). Intrinsic motivation is desired because the desire to learn comes from within the student, whereas extrinsic motivation is less desired, where the motivation is due to some outside force Hanus & Fox, 2015). In an empirical study of graduate students, Hew et al. (2016) find that students in the gamified course chose more difficult assignments and produced higher-quality artifacts than those in the nongamified course. We design assessment activities of the coursework with game mechanics following the suggestion of SDT on competence, relatedness, and autonomy.

Design Elements and the Course Choice
Considering the advice of many scholars on the need for careful design and alignment (de-Marcos et al., 2016;Fitz-Walter et al., 2017;Sobocinski, 2017;Aldemir et al., 2018), the ideal scenario for a course gamification design would be where the instructor can survey the students in advance to identify their needs and goals, thus customizing the gamified course design accordingly. In reality, however, most university students enroll in courses via an online system, and the course membership tends to change until the semester begins, as well as until weeks into the semester with drops and switches. Thus, the instructor rarely has a chance to meet or communicate with all of the enrolled students effectively until the semester begins. The alternative, then, is to identify the needs and goals that are vastly common among university students.
In higher education, students are exposed to high levels of stress and anxiety revolving their performance in assessments, such as quizzes, reports, and exams (Kapitanoff, 2009;Dahlström, 2012;Cantwell et al., 2017;Johanns, Dinkens, & Moore, 2017;Levine et al., 2017;. In this research, we apply gamification on the course element of assessment activities with a reward structure that connects them towards midterm and final exams to explore its effects on students' content knowledge and perceptions on satisfaction, experience, learning, and impact of teaching methods. The focus on student satisfaction, explored by Reinig et al. (2011), addresses the remark of Jassawalla, Sashittal, and Malshe (2009): "What is curiously missing in the rich body of research is the perspective of the student (p.43)." Finally, we decided to gamify the introductory operations and supply chain management (OSCM) course which is one of the most challenging courses to teach business students. Similar to Kong's (2019) motivation to choose a modeling and simulation course for game-based teaching, our choice of gamifying OSCM reflects our observations of business students being challenged by the analytical content (e.g., statistical process control, safety stocks, forecasting) in this course and the increased need to better engage students.

Experimental Design
For this study, two OSCM courses were designed. Both Section 1 (10 a.m.) and Section 2 (1 p.m.) were taught 3 times a week (Mon., Wed., Fri.) for 50 minutes each. Section 1 was designated as the experimental (gamified) group, while Section 2 was designated as the control (non-gamified) group. The two sections had the same instructor, course structure, learning objectives, assignments, quizzes, term projects, and exams. Section 1's curriculum had (i) a gamified structure of assessment activities, and (ii) an overlay of videogame nomenclature integrated to the assessment activities, following the application by Lieberoth (2015) and Stansbury and Earnest (2016).
Both sections were administered following a flipped classroom model (Herreid & Schiller, 2013;Asef-Vaziri, 2015), where brief lectures were delivered to students via pre-recorded videos, uploaded to YouTube. Students from both sections had access to the same videos at the same time. The length and content of videos were limited so that they function as a study guide rather than a full lecture. Students were required to complete class preparation by reading the textbook and watching the study guide video before the class meeting. Figure 1 depicts the progression of assessment activities in a flowchart. In the first weeks of the semester, students were assigned to teams by the instructor to ensure membership diversity in terms of major, gender, age, and nationality; also, team membership was maintained throughout the semester (Koppenhaver & Shrader, 2003).
In both sections, a short quiz of 5 questions was administered at the beginning of each class session as formative assessment of content knowledge and attendance-taking. The quiz questions tested knowledge of key terminology and concepts. After finishing each chapter, student teams were required to produce chapter reports, in one of the following formats: writing, presentation slides, infographic, or video. Teams were required to identify key concepts and topics from the chapter and apply the knowledge to a realworld context. In both sections, one individual research report per semester was

Gamification in Course Assessment Activities
Gamification was applied to this course by structuring assessment activities to build up towards the midterm and final exams with team rewards and by adding a layer of videogame-inspired nomenclature. Figure 2 depicts the gamified structure of assessment activities.
The quizzes were labelled 'Farming' and counted towards individual grades. Also, each team earned a ticket as a reward if all members of the team were present and scored higher than 60%, and the team average score is higher than 80%. Individual research report was labelled 'Quest'. Additional to individual grades and extra credit, a team ticket was rewarded for every additional individual report. The team chapter reports  The midterm exam and the final exam were labelled 'Season 1 Boss Raid' and 'Season 2 Boss Raid', respectively. The individual exam was labelled "singleplayer", and the team exam was labelled "multiplayer". For the multiplayer, teams had conditional access to unique advantages, labelled "power-ups", such as one textbook, one calculator, and one page of hand-written notes. These power-ups were available for teams to purchase as a result of the team's performance and member contributions in Farming, Missions, and extra Quests, thereby connecting the basic activities of watching lecture videos and reading the textbook with the exams. Before each Raid (exam), teams were given ample opportunity to win more tickets than needed to purchase all three power-ups. In the control group, in contrast, one textbook, one calculator, and one page of hand-written notes were readily accessible to teams during the team exam. Table 1 provides a comparison of how gamification was applied to assessment activities in the course, as well as the theoretical focus within SDT in each element.
Farming (quizzes) promotes competence through positive feedback. The quiz items ask basic conceptual questions, which is easy to answer if the student watched the video and read the textbook. The team reward system from Farming promotes relatedness. Missions (team chapter reports) promote all three focal areas of SDT: competence through adjustment of difficulty adequate to chapter progression (start with summaries and move on to case studies), relatedness through teamwork and team reward, and autonomy through meaningful choices for report format and case topic.
Quest (individual research report) promotes autonomy through meaningful choice in topic selection, number of additional reports, and timing of submission, as well as relatedness through contribution to team reward for additional reports. Boss Raids (exams) promote competence through the progressive difficulty of questions, relatedness through collaborative exam, and autonomy through meaningful choice regarding which Power-up to purchase with the team reward tickets.

Participants
The study took place at a university in the northeast of the United States in the course of one semester in the fall of 2018. 62 undergraduate students participated, divided into two sections of 33 (Experimental, gamified) and 29 (Control, non-gamified) by enrollment. In order to control for selection bias, the online enrollment system displayed the same generic course description, standard to the College of Business, for the two sections. This way, students were not exposed to the pertinent experimental conditions and manipulations before enrolling. Information on the course design was only given after the semester started. OSCM is an introductory course that is required for all business major students of the university and is also open to students of other majors that satisfy the prerequisites. 19.35% of the students in the current study were female.

Measures and Procedures
We tested the effect of assessment gamification on student's content knowledge, satisfaction with learning method, course experience, perception of learning, and impact of teaching techniques, following the survey methods of Stansbury and Earnest (2016) and Reinig et al. (2011), as detailed below. Content knowledge, course experience, perception of learning, and impact of teaching techniques were replicated or modified from Stansbury and Earnest (2016). Satisfaction with learning method was modified from Reinig et al. (2011).

Content knowledge measures
The content knowledge of students was calculated as the mean score of each student's total quiz and exam mean scores. At the beginning of every class session, a 5-item quiz was given to students. Throughout the semester, a total of 24 quizzes were administered.
The quiz items were either short-answer or multiple-choice questions on basic concepts of the session topic, which students were required to have studied in advance with the textbook and lecture video. Items on the quizzes were adapted from the test bank provided by the authors of the course textbook and were identical for both groups throughout the semester. Student performance on the quizzes was measured as a percentage of right answers. Because the quiz had a double purpose of attendance-taking, the quiz grade penalizes absenteeism, resulting in an inaccurate measurement of student's content knowledge. We resolved this issue by only accounting for quizzes taken at the time of class, excluding absentees from the average.
The midterm and final exam were administered as a 2-stage collaborative test with different sets of questions for each stage. Individual student performance in exams was measured as a percentage of right answers. The mean exam score was averaged with the mean quiz score to create a single variable, "Knowledge".

Formative perception measures
In the latter half of the semester, we conducted formative surveys in 8 different time points assessing student's perceived satisfaction with learning method, perceived experience, perceived learning, and perceived impact of teaching techniques used in the course. For the formative survey instrument, a 4-item perceived satisfaction survey from Reinig et al. (2011) was used. Also, a 4-item perceived experience survey and a 7-item perceived learning survey were adopted from Stansbury and Earnest (2016). A 7-item perceived impact of teaching method survey was adapted from Stansbury and Earnest (2016), with modifications to fit the context of course. Formative survey time points occurred over a 6-week period between the 10 th and 15 th weeks of classes. The analysis used 176 accumulated surveys from the experimental group, and 167 accumulated surveys from the control group (N = 343). Table 2 provides a full list of survey items, constructs, and descriptive statistics.
Items for perceived satisfaction with the learning method were adapted from the survey design of Reinig et al. (2011) and stated, "I feel satisfied with the learning method used in this class," "I liked the learning method used in this class," "I would like to use this learning method in other classes," and "I was happy with the learning method used in this class". Responses were measured on a 5-point Likert scale with end points: 1 = strongly disagree to 5 = strongly agree. All four items loaded onto a single construct. The items on this measure were highly reliable (α = .955). The mean score for the four satisfaction items was calculated as perceived satisfaction with learning method variable, "Satis".
Items for perceived course experience were modified from the survey design of Stansbury and Earnest (2016) and asked student's honest feelings on their course experience in six areas: motivating, engaging, fun, boring (inverse measure for fun), challenging, and relevant. Responses were measured on a 5-point Likert scale with end points: 1 = strongly disagree to 5 = strongly agree. The items on this measure were highly reliable (α = .813). The mean score for the six experience items was calculated as perceived course experience variable, "Exper".
Items for perceived learning were adapted from the survey design of Stansbury and Earnest (2016) and asked, "When comparing this course to my traditional courses, I would rate this course as being…" followed by 7 items including "motivating my learning" and "reinforcing key concepts". Responses were measured on a 5-point Likert scale with end points: 1 = strongly disagree to 5 = strongly agree. The items on the perceived learning measure were highly reliable (α = .934). The mean score for the seven perceived learning items was calculated as perceived learning variable, "Learn".
Items for perceived impact of teaching technique measures were adapted from the survey design of Stansbury and Earnest (2016) and asked students how much their learning in the course was aided by seven teaching techniques: classroom discussions, team chapter reports, quizzes, professor, exams, lecture videos, and textbook.
Responses were measured on a 7-point Likert scale with end points: 1 = not at all to 5 = extremely. The items on the perceived impact of teaching technique measure were treated as individual variables, following the suggestion of Stansbury and Earnest (2016).

Summative perception measures
We also conducted a 27-item summative follow-up survey for the experimental section at the end of the semester, after the final exam (N = 23). The summative perception survey contains 19 items on their overall perceived experience with the gamified course design (e.g. I enjoyed the gamified design of the class, the gamified design kept me engaged), 6 items on perceived efficacy of individual game elements (e.g. alteration of terms, reward system, and meaningful choices), as well as items on perception of videogames and learning (e.g. collaboration and teamwork, creativity and problemsolving) and learning style preference (competitive, cooperative, and individual). In the overall perceived experience measure and perceived efficacy of individual game elements measure, responses were assessed using a 5-point Likert scale with end points: 1 = strongly disagree to 5 = strongly agree. The perception of videogames and learning measure provided a list from which to select all that applies. The learning style preference measure provided a list from which to choose only one that applies. The items on the summative follow-up overall experience measure were highly reliable (α = .95) and were explored to provide descriptive insights. Table 3 contains a full list of items and descriptive statistics.

Quiz scores
Results of a two-independent samples t-test (Table 4) shows that the mean score for quizzes taken differs between the experimental (gamified) group (M = 81.86, SD = 7.74, n = 33) and the control (non-gamified) group (M =86.39, SD = 7.67, n = 29) at the .05 level of significance (t = -2.31, df = 60, p = .024, 95% CI [-0.085, -0.006]). On average, students in the non-gamified control group scored higher in the quizzes than those in the gamified experimental group, as seen in Figure 3.

Exam scores
Scores from the midterm and final exams were divided into individual and team score from the two stages. A two-independent samples t-test shows that the total mean score for both exams differs between the experimental group (M = 70.71, SD = 6.59, n = 33) and the control group (M = 76.41, SD = 7.46, n = 29) at the .05 level of significance (t Total 67.12 9.77 74.31 9.28 -0.120 -0.023 -2.96** 60 .004 *p < .05. **p < .01. ***p < .001. = -3.20, df = 60, p = .002, 95% CI [-9.269, -2.133]). On average, students in the control group scored higher in the exams than those in the experimental group. Item-level analysis of the t-test on exam scores (Table 4) shows that students in the experimental  (Table 5).

Perceived satisfaction with learning method
Results of the two-independent samples t-test (Table 5) (Table 6) show a significant between-group difference in satisfaction (F = 5.776, p = .017). There was no significant difference associated with survey sessions (SRV, proxy for time, p = .806) or the interaction of group (Sec) and survey session (p = .504).

Perceived course experience
Results of the two-independent samples t-test (Table 5) (Table 7) show a significant betweengroup difference in experience (F = 4.323, p = .038). There was no significant difference associated with survey sessions (SRV, proxy for time, p = .441) or the interaction of group (Sec) and survey session (p = .947).

Perceived learning
Results of the two-independent samples t-test (Table 5)  The fixed effect ANOVA results (Table 8) show a non-significant between-group difference in perceived learning (F = 2.421, p = .121).   Group differences in other variables were not significant. In the experimental group, students reported highest impact of textbook, followed by professor, quizzes, and classroom discussions. Students in the control group report highest impact of professor, followed by classroom discussions and quizzes.
A significantly higher mean score for impact of textbook supports the course design of utilizing gamification to motivate students to read the textbook before coming to class.

Summative Perceptions
The summative survey analysis with 19 items on gamification perceptions (α = .945) reveals that students in the experimental section had favorable perceptions towards most individual items (Table 3). It is notable that the highest mean score is reported on item GS12, "I enjoyed being a part of a team" (M = 4.09), followed by two other items regarding social dynamics: items GS4, "Being part of a team motivated me to study" (M = 3.96) and GS14, "Gamified design allowed me to collaborate with others in the class" (M = 3.96), because the design of gamified assessment in the course had a heavy focus on social interactions and teamwork.
Students in the experimental group also reported overall positive perceptions towards individual game elements used in the course (Table 3). Again, students reported highest mean scores on item GE4, "teamwork and team dynamics in tasks and exams" (M = 3.71), followed by item GE3, "reward system (tickets for power-ups, M = 3.68), which is also a social gamification element and the core driver for the gamified structure.

Academic standing
To check for a potential bias from student's previous academic standing on content knowledge, additional analyses were conducted to explore the possible effect of gamification on content knowledge, after taking student's cumulative grade point average (GPA) into account. A Scatter plot indicated a low degree of positive correlation between student's GPA and content knowledge. However, there was no significant difference between the two group's average cumulative GPA when the semester started.
Thus, we can rule out the possibility that the difference in content knowledge and some perceptions between the experimental and control group were driven by previous academic standing.

Gender
Although the basic premise of gamification and SDT is that demographic differences

DISCUSSION AND SELF-REFLECTION
The results reported in Section 4 were surprising to us as we expected gamification of the OSCM course assessment element to have a positive impact on outcome variables.
While there have been a few critiques of gamification in the literature, they mostly focused on the use PBL as a stock approach and its negative effects on students' intrinsic motivation (Deci, Costner & Ryan, 1999;Aldemir, Celik & Kaplan, 2018). Considering the advice of these critical studies, we avoided the classical PBL approach, gamified only one course element and used no online platform to design a generalizable course gamification experience. The surprising findings of our study make significant contributions to the literature by demonstrating a dark side of gamification in a classroom setting and by exploring the effects of selectively gamifying a course design element (assessment) with no online platform involved. Below, we will interpret our findings and reflect on our experiences to assist other professors design their own course gamification projects.
First, we would like to share a few positive observations of gamification that are more aligned with the literature. Students in the experimental group were vocal about their excitement and motivation towards the course. In more than one instance, a group of students in the experimental group would approach the instructor and express their excitement for the course, contrasting their frustration with another course that was not gamified. In an anonymous written survey, administered in the 9 th week, students responded with some positive comments such as "I really enjoy the videogame teaching style", "I like the videogame format", "the farming forces you to study almost daily", "I like that I have to rely on myself to teach myself", "I believe group work is very effective", and "I like how you offer the various ways we can do the mission reports because it helps me learn".
In addition, an analysis of perceived impact of teaching technique reveals that students in the gamified experimental group perceived the textbook as the most impactful resource, while those in the control group perceived the professor to be the most impactful element in their learning (Table 9). Student response to the instructor's proprietary midterm and final survey question on student perception of the textbook was overwhelmingly positive in the gamified group, whereas several in the control group expressed indifference or negative perceptions. This suggests that our iteration of assessment gamification motivated students to read the textbook before class to be ready and resulted in them being more self-regulating, which is associated with higher intrinsic motivation in SDT . Controlling for the effect of the flipped classroom design, which was identical across sections, gamification of assessment activities seems to have had positive effect on making students read the textbook.
Our statistical results, interestingly, suggest that gamification of assessment significantly decreases student's content knowledge. The declining linear trend in the quiz scores from the experimental group is consistent with the observations made by Koivisto and Hamari (2014), where perceived usefulness, enjoyment, and playfulness are found to diminish with time interacting with gamified system. It is possible that gamifying assessment had some novelty value in the beginning of the semester resulting in higher motivation and engagement, but as students became accustomed to the course design and perceived the tasks as equally rudimentary as those in any other course, the novelty value may have faded and even harmed student performance in quizzes and exams. Lieberoth (2015) finds that adding a playful frame to tasks ("shallow gamification") takes away the grit and output orientation of more goal-oriented work, which may further explain the declining trend in the content knowledge of the experimental group.
Another reflection comes from the reward system. We used common rolled-up drink tickets as reward/currency in the gamified system. Although students appreciated being awarded tickets for their quiz and team assignment performances, the items themselves held little sentimental value. Regarding the effect of rewards on intrinsic motivation, the literature provides controversial viewpoints. Deci and Ryan (2000), based on findings of a meta-analytic review of studies on the effects of extrinsic rewards on intrinsic motivation, argue that tangible rewards that are expected and taskcontingent have negative impact on intrinsic motivation (Deci et al., 1999). In fact, Deci et al. (1999) find that all contingent tangible rewards, including monetary, significantly undermined intrinsic motivation, categorically refuting the argument of Eisenberger and Cameron (1996) that the detrimental effects of rewards is mostly a myth (Deci et al., 1999). However, these undermining effects of intrinsic motivation are only present in interesting tasks, not dull or boring ones (Deci et al., 1999). On the other hand, Aldemir et al. (2018), while extending the findings of Deci and Ryan (2000) with application on higher education, stress the need for a continuous and systematical reward distribution, with tangible items that are inexpensive but hold high sentimental value, to avoid students suddenly losing their motivation to continue. Enhanced aesthetic and sentimental value of the reward objects may enhance the prolonged engagement with the gamified system. The value of the reward was also found to be a major modulating factor in enhancing episodic memory (Mason, Farrell, Howard-Jones, & Ludwig, 2017), making the association of high-value reward with an immersive experience an effective mechanism for gamification (Mullins & Sabherwal, 2018).
A clearer disparity regarding the reward system exists between the unit of assessment and the unit of rewards. While the quizzes were given as individual assessments, the tickets were awarded for the team performance. Typically, teams assigned one member to collect and keep the awarded tickets (typically in a zip-lock plastic bag) while the rest of the team had no access to them. It appears that setting the team as the unit for rewards for individual assessments may have hindered the "slicing up" of motivating effects to the individual level. We suggest that rewards should, first and foremost, benefit the individual before the team in order to retain prolonged motivation and engagement of all team members. Considering the focus of SDT on the relationship between competence and intrinsic motivation , proper reward, adequately valuable to the perception of students, should be given to individuals in addition to the teams.
Another meaningful observation comes from the individual and team score differences between the two groups. A closer look at the content knowledge measurements reveals that the gamified group scored significantly lower than the control group in quizzes and team exams, but not in individual exam scores. These results suggest that gamification adversely affected student's team performance, while it did not affect their individual performance in the summative knowledge assessments. (2015) find that encouraging competition and social comparison harms intrinsic motivation, which in turn results in lower exam scores. Although their study environment uses an online leaderboard and badges, resulting in some degree of difference from the present study, the settings in present study may still invoke similar social comparison and competition. Specifically, after completing the quiz feedback, the instructor collected the quiz sheets by team and immediately awarded tickets to teams that qualified. Also, the instructor periodically asked teams to report the current quantity of tickets accumulated by each team.

Hanus and Fox
The insights gained from the quizzes and exams show that the present gamification design involving the assessment of the course is, at best, insufficient to deliver a positive impact on student's content knowledge. This may be due to the long duration of the gamified experience, the simplistic reward system that rewards the team for individual work, and latent social competition that led to lower intrinsic motivation.
It is apparent that the design needs to be improved to encompass more elements than assessment in order to retain student engagement and motivation, which will lead to enhanced content knowledge. Also, In the University's official course evaluation survey, one student from the experimental section commented: "Harder than most courses because students were expected to teach themselves. I wish the professor taught before we were quizzed". This suggests that the course's overall structure and the sequence of activities may interact with gamification.

Student Perceptions
The results of the formative perceptions survey suggest that assessment gamification had a significantly adverse effect on student's perception of satisfaction and course experience. However, perceived learning did not show a significant between-group difference. This suggests that, although our version of assessment gamification may have had a negative effect on student's satisfaction and course experience, they still felt like they were learning from the course. Also, it is important to note that, despite the significantly negative results in satisfaction and course experience in comparison with the control group, the survey results show that the gamified group is still highly satisfied (M = 4.04 / 5) and having an overall positive course experience (M = 3.81 / 5) (Table   5).
These results and observations, combined together, suggest that students in the experimental section, as the initial hype and novelty value faded, perceived the gamified activities for coursework as what they actually are: coursework. Thus, they may have perceived slightly less satisfaction and poorer course experience overall due to certain degree of disillusionment as well as the negative effects of the reward system, but it did not result in a detrimental decline in satisfaction or experience. Also, they developed a sense of self-regulation and autonomy in reading the textbook before class. The cooperative social emphasis of gamification that builds every assessment activity up towards the exams also created a sense of accountability and team membership, apparent in the high mean scores in social aspect items of both formative perceptions survey ( Table 2, IT1, IT2) and summative gamified course survey (Table 3, GS4, GS12, GS14, GE3, GE4), although they did not manifest as higher team exam scores. These results are consistent with the emphasis on the social aspect of gamification in the literature (Hamari & Koivisto, 2015;de-Marcos et al., 2016;de-Marcos et al., 2017).

Gamification of Assessment
In the establishing work for SDT, Deci and Ryan (2000) refer to studies of Heider (1958) and de Charms (1968) on internal perceived locus of causality (PLOC), as well as additional studies that showed intrinsic motivation is undermined by threats, surveillance, evaluation, and deadlines because they shift the internal PLOC (I-PLOC) to external PLOC (E-PLOC). These studies are grounds on which Deci and Ryan (2000) argue the importance of satisfying the need for autonomy and sense of self-initiation for increased intrinsic motivation that can lead to greater creativity and better problemsolving.
In the present study, although choices were given for some components, such as team report format, the quizzes and exams were mandatory, and student performance in these mandatory activities had direct and heavy consequences in their grades. This is an issue especially prominent in the context of higher education, because it is nearly impossible to completely eliminate certain factors that may control or manipulate students in the current design for undergraduate-level coursework. In order to achieve the desired level of choices and autonomy, perhaps assessment shouldn't be the only element of the course to be gamified.

CONCLUSIONS AND FUTURE WORK
We are surprised to find that gamification of assessment activities of a course has negative impact on content knowledge and student perceptions. Our design was intended to explore the benefits that gamification can deliver to students when applied to course assessment. This design primarily answers the growing call for empirical studies on specific application contexts, as well as for isolating individual design elements. We did observe a heightened initial enthusiasm from the students for the gamified design, as well as the effect of gamification on promoting textbook reading before class. However, those observations did not lead to significant improvements in content knowledge and student perceptions, and even resulted in significantly inferior results. Our negative results suggest that gamification can hurt content knowledge and student perceptions when applied only to assessment of coursework. Instead of simplistic game mechanics applied to narrow areas of the coursework, the gamified system must encompass a multitude of mechanics and elements that can entertain various needs of students. This principle should apply across contexts: education, marketing, training, management, etc. This does not mean that gamification must be applied to all aspects of coursework to be successful. Finding the optimal mix of gamification in the coursework would be a valuable future study.
As gamification research matures, there has been a subtle but growing call for elements of games and videogames that weren't emphasized in previous iterations of gamification in either business or academia. These are calls for elements that contribute to the ever-elusive aspect of fun; elements on which videogame developers and game makers spend enormous time and resources to develop, but somehow got lost in the transplantation of games to corporate and educational contexts: narrative and play style are among these elements. Narrative not only creates a more coherent experience, but also facilitates the need to be part of something bigger than oneself, as addressed by Mcgonigal (2011) in the opening quote of this study. an interdisciplinary team spanning from business, computer science, literature, theater, etc. may work best to design, implement, and execute a gamified coursework with rich, epic narratives and carefully placed elements and mechanics to deliver a fun, engaging, and effective learning experience.
This study extends the growing literature on gamification in higher education by implementing gamification to a narrower area of learning experience, focused on the single course element: assessment, and used no online platform. Our analysis and observation lead to a conclusion that gamification is at best insignificant when applied only to assessment, and at worst may significantly harm student perceptions and academic performance.

Limitations
This study is limited in several ways. First, due to an iterative process of administrative approval for human subject research, the formative surveys were delayed until the 10 th week of the 16-week semester. As a result, while assessments in content knowledge span the entire semester, the data from the formative perceptions surveys are limited to the later third of the semester. Also, due to confidentiality and anonymity requirements, the surveys contained no identifiers of individual students. Therefore, we were unable to treat the survey as repeated measure to explore the effect of assessment gamification on individual student's perceptions over time. Although we do acknowledge this to be a major limitation, we opted for anonymity to avoid the possibility of students giving positively biased answers in the surveys due to coercion.
The quiz/exam scores and survey results of this study provide an interesting interpretation that lead to more questions. The controversial effect of rewards in a

ABSTRACT
College students that major in business strive for opportunities to apply their learning to the real world to be better prepared for their professional lives after college.
Newspaper op-eds, dedicated to commentary essays from outside of a newspaper, function as a platform for the public to exchange opinions. We designed an op-ed writing project with the following learning goals: 1) to educate students on how to write an op-ed, which is a novel and unfamiliar writing style for most students; and 2) to educate students on how to publish their opinions in a newspaper, which includes how to interact with journalists.
We present an op-ed writing project for students enrolled in two sections of a Global Supply Chain Management (SCM) course. The project was given as a team project, and entailed writing an op-ed on a current topic related to global SCM and submitting them for publication in local and regional newspapers. We conducted a survey that included qualitative and quantitative questions on the perceived greatest takeaways, greatest challenges, and feelings on the project.
Seven student teams successfully published their op-eds in local and regional newspapers. Students were excited to learn they, too, can participate in the exchange of opinions in the public domain via op-eds. Many expressed content in learning a new style of writing, deviating from the typical academic report writing. Students perceived that this project helped them learn the course materials better, and their sentiment towards the project was predominantly positive.
Considering multiple student op-eds successfully published in local and regional newspapers, along with the positive survey results aligned with the learning goals of the op-ed writing project, we consider that the goals of this project were successfully achieved.

INTRODUCTION
The "deepest responsibility" of the newspaper, Oakes wrote, was "the same responsibility ... that the college has for its students -the responsibility of making them think." Thus, a fundamental purpose of the editorial page was "to question, to debunk." "Diversity of opinion is the lifeblood of democracy," Oakes contended in a 1954 speech.
"The minute we begin to insist that everyone think the same way we think, our democratic way of life is in danger." John B. Oakes (N.Y.

Times editorials editor that created the op-ed page), Requoted from
Socolow (2010) The outside commentary page of a newspaper, commonly referred to as op-eds for being on the "opposite of editorial page", publish opinion essays on various topics, even if the opinion is not aligned with the newspaper's own. Op-eds in newspapers, thus, function as "a forum for the exchange and clash of ideas...covering a whole broad range of intellectual interests" (Socolow, 2010, p. 286 We demonstrate a case of integrating op-ed writing to upper division supply chain management (SCM) courses at a northeast U.S. university. The op-ed writing project was given as part of a semester-long team research project. Student teams were required to conduct independent research on emerging SCM topics, deliver a 30-minute presentation in the classroom, and write an op-ed article on their assigned topic. The learning goals of the op-ed writing project are: 1) to educate students on how to write an op-ed, which is a novel and unfamiliar writing style for most students; and 2) to educate students on how to publish their opinions in a newspaper, which includes how to interact with journalists. Channeling students to express their opinions on contemporary business/SCM topics for the benefit of the community, this project also fosters students' critical thinking capabilities and sense of social responsibility.
These contributions are aligned with the goal of nurturing active thinkers that participate in the process of democracy by speaking up in the public fora on issues that are important to their communities and professions, thus returning benefits of their university education to the public.
We also demonstrate an alternative implementation of the op-ed writing project with a different approach for the topics. Instead of an array of contemporary SCM topics, the alternative implementation uses a case study based on a local supply chain disruption.

LITERATURE REVIEW
The benefits of incorporating op-ed writing in coursework have been demonstrated by various scholars. J. C. Hall and Podemska-Mikluch (2015) showed that op-ed writing is the most conducive way of encouraging students to be sensible and active members of civil society, as well as active advocates of economic reasoning. Schneller (2016) demonstrated that op-ed writing can be used to help students develop and articulate their personal perspectives about environmental and sustainability issues. Calavita and Krumholz (2003) identified op-ed writing as highly visible public participation that can inform and invigorate teaching and help engage students. Poronnik and Moni (2006) found that although students perceived op-ed writing to be challenging, they also believed their ability to effectively communicate their knowledge to nonprofessional audiences in writing was improved after explicit teaching and feedback.
Our implementation of op-ed writing to coursework is based on active learning pedagogy. Active learning is defined as any instructional method that engages students in the learning process (Prince, 2004). It requires students to do meaningful learning activities and think about what they are doing (Prince, 2004;Bonwell and Eison, 1991). Bonwell and Eison (1991)  Bringing the active learning pedagogy into the social responsibility domain, Özpolat, Chen, Hales, Yu, and Yalcin (2014) required students to design public service announcements in support of USAID's smart compassion campaign to address unsolicited material donations. Wentzel (1991), upon reviewing the literature on social responsibility and academic achievement, argues that "student social responsibility is not only a valued outcome in and of itself, but that it can be instrumental in the acquisition of knowledge and development of cognitive abilities (p.16)". Kingston, Maccartney, and Miller (2014) identify social responsibility as an integral element of preparing students for global citizenship.
The implementation of our op-ed writing project is also based on team-based learning (TBL) (Parmelee & Michaelsen, 2010;Sweet & Michaelsen, 2012;. TBL focuses on small group interaction more heavily than any other commonly used instructional strategy (Michaelsen and Sweet, 2008). With TBL, group work is central to exposing students to, and improving their ability to apply, course content (Michaelsen and Sweet, 2008). Students work in small groups to finish the op-ed writing project, and the interaction between students will improve their ability to apply the course content.
The remainder of this brief consists of a detailed layout of the project in the two classes at a northeast U.S. university. After the discussion of the case, we discuss the insights from various sources, along with specific suggestions for improved implementation.

PROJECT OVERVIEW
The implementation of the op-ed writing project took place in the spring semester of 2019 at a northeast U.S. university. Two sections of an upper division SCM course were taught by the same instructor on Tuesdays and Thursdays, at 09:30 AM (Section 1) and 11:00 AM (Section 2) for 75 minutes. The op-ed writing project was part of a semester-long team research project, which required student teams to (i) conduct independent research on an emergent SCM topic, (ii) make a 30-min. class presentation, and (iii-a) write an op-ed on the topic with the team's opinion and submit it to a local/regional media outlet or (iii-b) write a 10-page research report. The class presentation served as a demonstration of independent learning and research as well as team coordination. The op-ed/report writing complemented the research project as the written deliverable. The research project had 30 percent weight in the student's final grade -25 percent on self-initiated learning and class presentation, and 5 percent on the op-ed writing/research report project.
In the first week of the semester, the instructor assigned students into teams of four, ensuring that every team had at least two different business majors. From a menu of 12 topics provided by the instructor, student teams identified their top three choices and were assigned to one. The research topics were related to either cutting-edge supply chain technologies or contemporary global supply chain management. A full list of research topics is provided in Table 10. In Section 1, all teams were required to write an op-ed of 600-700 words as a written deliverable of the research project. In Section 2, teams had a choice between an op-ed of 600-700 words and a 10-page team research report. The research report was meant to be a written summary of their presentation, including their research process and findings. Because the instructor deemed this to be a relatively simple task, it was determined that the time and effort required for writing a 600-700-word op-ed, which was a novel writing style for most students, was comparable to that required for a 10-page research report. Interestingly, however, all teams of Section 2 opted for the op-ed writing project. All other conditions were kept identical between sections. At the end of the semester, questionnaires were distributed to students for data collection.
The questionnaire included qualitative and quantitative items.

Participants
Section 1 had 44 students, while Section 2 had 49. All enrolled students were either juniors or seniors, and most were SCM or Marketing majors. Among the enrolled students, 56 were male and 37 were female. Table 11 shows the demographic breakdown of the students.

Milestones
In both sections, team presentations on research topics were spread out from the third week to the last week of classes. A first draft of the op-ed was due in the 9 th week. The TA provided feedback within a week, and the updated second draft was due in the 11 th week. Another round of review and feedback was provided in the same week, and a revised final draft was due in the 12 th week. After a brief last feedback in the same week, teams were required to submit the op-ed to a local/regional newspaper by the end of the 13 th week. If the first target newspaper rejected the manuscript, teams submitted to another newspaper by the end of the 14 th week. Table 12 provides a list of milestones.

In-Class Introduction of Op-Ed Writing
While the topic selection and classroom presentation portion of the team research project was guided and graded by the principal instructor who is an associate professor, the op-ed writing portion was guided and graded by a graduate teaching assistant (TA) at the doctoral level. The TA made a 20-minute in-class presentation in the fourth week of the semester, introducing the purpose, function, and format of oped writing. The detail of the op-ed writing project within the team research project was also covered. Three online articles (Omang, 2008;T. Hall, 2013; Duke University, 2019) on op-ed writing guidelines were shared with the students on the syllabi and during TA presentation. Also, the TA introduced a few op-ed articles written and published by previous semester's students to demonstrate not only the structure of an op-ed but also the feasibility of student publications.

Grading
Regarding what constitutes a good op-ed article, we referred to resources available online (Omang, 2008;T. Hall, 2013;Duke University, 2019) to identify several important components: a strong hook to grasp the reader's attention at the beginning, a strong argument with personal voice, demonstrated personal connections to the readers, plentiful evidence supporting the argument, specific recommendations or policy suggestions, and a concsise writing with 600-750 words. Combining these components with the university's standards for academic writing projects (e.g. grammar, punctuation, structure, etc.), we created an op-ed grading rubric (Appendix A).

ASSESSMENT OF THE PROJECT
The Experience: TA Review, Submissions, and Publications As specified in Table 12, student teams submitted their first draft op-ed in MS Word format at the end of the 9 th week. The TA received the submissions via e-mail from each team and provided written feedback in the form of edits and electronic memos on the MS Word document. The first draft feedback was predominantly on conformity to the op-ed format, since the majority of teams were following the familiar academic report format. A specifically prevalent issue was the lack of opinion in the writing, since summarizing their research project alone tended to take up most of the 600-to-700-word limit. Another widespread issue was the lack of a strong hook in the first paragraph that grasps the attention of the reader. The first round of review and feedback for the 23 manuscripts took the TA a full week to review, and the student teams had a week to work on the revision and resubmit the second. The TA also posted general feedback in the Announcement board of the learning management system (LMS), along with instructions for the second draft submission. TA's first general feedback is provided in Appendix B.
The second draft submission saw significant improvement in several teams, yet many teams were still stuck at the academic format. Some made a transition into a news article, establishing relevance between the topic and local/regional/national implications but still lacking a strong opinion. Prevalent issues in the second round also included the first paragraph not containing the main point, ambiguous and broad suggestions and solutions, and too much detailed information. The second round of review and feedback for the 23 manuscripts took the TA four days to review, and the student teams had four days to work on the revision and resubmit the final draft to the TA. The TA also posted a general feedback in the Announcement board of the LMS, along with instructions for the final draft submission. Preparing students for the final stage, the instructor and TA provided a list of local media outlets for the students to consider. The second general feedback is provided in Appendix C. At this time, the TA also provided a sample cover letter for submission to the editor. The sample cover letter is provided in Appendix D.
The final draft submission saw significant advances in many team's manuscripts, and the reviews were finished in one day. The TA gave teams specific instructions for formatting their manuscripts and submitting them to one local/regional newspaper within 24 hours. Students were instructed not to reveal the fact that their manuscript is part of a course project but take full ownership of the submission. The initial submission instructions are provided in Appendix E. A week after the initial submission, the TA posted instructions for withdrawal and resubmission to another newspaper for those teams with no acceptance. Students were instructed to send a letter of withdrawal to the editor if the editor didn't respond to their initial submission within 12 days. The TA graded each team's op-ed manuscript based on the op-ed grading rubric developed from multiple resources on op-ed writing (Omang, 2008;T. Hall, 2013;Duke University, 2019) and the university's academic writing rubric. The average score of Section 1 was 90.1 out of 100, and that of Section 2 was 92.8. The average of both sections at 91.4 signifies that the first learning goal of educating students how to write an op-ed was achieved.
14 out of 23 manuscripts were initially submitted to a prime regional newspaper. Three teams initially submitted to a regional business newspaper. Three other teams submitted to local newspapers, and one submitted to the university newspaper. Out of the initial submissions, three teams were accepted and published in the business newspaper and two local newspapers. The second submission (after withdrawal from the first newspaper) was dispersed to various local newspapers.
Three more teams published in two local newspapers. We provide hyperlinks to the published student op-eds in Appendix F. Seven student op-ed publications in local and regional newspapers and a practitioner journal signifies the second learning goal of educating students how to publish their opinions in a newspaper was achieved.

Qualitative Survey
Three qualitative questions were asked to explore student perceptions on the greatest takeaway and the greatest challenge from the op-ed writing project, as well as their feelings for the op-ed writing project. Two techniques and procedures were adopted from Kock, Verville, and Garza (2007) to code the qualitative responses. The first technique identified patterns across the textual data. Responses were categorized into several patterns, and the frequencies of the patterns were calculated as percentages (Miles & Huberman, 1994;Yin, 2017). The second technique employed was content analysis. We applied manifest (semantic) content analysis and latent (inferred) content analysis (Wilson, 1989) on the patterns identified by the first method. The next sections discuss the results of the qualitative data analysis of the textual responses for the questions on the greatest takeaway, greatest challenge, and feelings towards the op-ed writing project.

Takeaway
After iterative rounds of discussions among the authors, responses to the question on the greatest takeaway were coded into six categories (Voice, Learning, Community, Publication, Style, and Others). The Voice category includes illustrative quotations on how the op-ed writing project helps students to express their opinions on a news outlet. The Learning category includes illustrative quotations on how the project helps them learn the topics deeper. The Community category includes illustrative quotations on how the project helps them connect and contribute to their community. The Publication category includes illustrative quotations on how the project helps them learn how to publish on a newspaper. The Style category includes illustrative quotations on learning how to write an opinion column, which for many of our students was a novel style of writing. All those quotations that don't fall into the five categories specified above were coded into the Others category. Table 13 summarizes the illustrative quotations of each category. quotations on the challenges of making meaningful connections between the generic topic and the regional context. The Iterative review process category includes illustrative quotations on the challenges of going through the TA-guided revision Table 14. Summary of student responses to greatest challenge process. All the quotations that don't fall into the six challenge categories specified above were coded into the Others category. Table 14 summarizes the illustrative quotations for each category.

Feelings
The responses to the question asking how students feel about the op-ed writing project were first categorized for overall feeling, in Positive (57.81%), Neutral (26.56%), and Negative (15.63%). To validate the categorization, the authors ran a sentiment analysis with R-studio. The responses were compared to the "bing" and "nrc" lexicon, which identify specific, pre-determined words in responses as criteria for categorization. The   Table 15 on a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree). The percentage of students that agree or strongly agree to each statement is shown in the far-right column of Table 15. In sum, over half of students surveyed agree that working on the op-ed as a course project helps them to learn the topics in more depth (e.g. 63% of the students agree and strongly agree that writing the op-ed improved their understanding on the research topic)

DISCUSSION
Designed as part of an active learning research project, the op-ed writing project offered many benefits to students. It is our observation that this project helped students develop their own voice on a topic related to their profession, learned a new way of writing, practiced teamwork, and learned how to publish on a newspaper. In this section, we discuss the benefits of this project as well as how it can be further improved.

Voice
Students expressed greatest excitement in realizing that they can also participate in the public opinion exchange through writing op-eds. The op-ed writing project, with its instructional elements, supplemented the research project by promoting an outlet for contributing student's learning to the community and society. Many students mentioned that they realized it was so easy to submit an op-ed and their own opinion can be valuable. Writing an op-ed brings students out of the typical role of receiving knowledge, enabling and empowering them to express and construct their own opinions and solutions.

Learning
Student comments on their takeaway on learning indicate that writing an op-ed enabled them to take a deeper look at the topic that they would have otherwise disregarded, and to apply the knowledge making connections between classroom knowledge and reality. These remarks are aligned with the benefits of active learning: a deeper understanding of course concepts and a positive attitude toward learning (Salemi, 2002).

Novelty
Many students expressed excitement in doing something for the first time and learning a new writing style and structure that aims to grab public attention. The difference in format, e.g. heavier focus on the opinion than on information, and the ability to use first person, were mentioned as takeaways.

Teamwork
Learning from the interaction with peers is one of the major benefits of active learning. Thus, the research project and the op-ed writing project were designed as team tasks. The teamwork component is also intended to enhance the student's soft skills of exchanging opinions among teammates, making compromises and negotiating a common ground, and collaborating for a common goal. The instructor and TA witnessed students actively engaging in team discussions and debates through the entire course of the semester, and the resulting team presentations and op-eds were indicators of success for such goals. However, the process for the students was apparently not easy. Student comments indicated hardship in forming a single strong opinion among four individuals with diverse views. Multiple student responses explicitly suggested assigning less people to a team -teams of two instead of four or five, or even making op-ed writing an individual project.

Timing
Activities for the op-ed writing project were concentrated in the second half of the 14- week semester, whereas team research presentations were spread across those 14 weeks. Thus, the disparity of timeline may cause several issues. First, student teams that finished their research presentation earlier in the semester may be detached from their topic by the time they were required to write the op-eds. Second, the grading TA must process multiple waves of overwhelming load of student manuscripts within a short period of time. Third, the resulting student manuscripts tend to be concentrated to a few news outlets, potentially biasing the editor's decision towards rejecting most of them. Introducing the op-ed writing project early in the semester and initiating each team's writing process after their presentation may enhance retention of student interest, avoid concentrated workload for the grader, and enhance the chance of being accepted for publication.

Choice
Teams of Section 2 had a choice between writing an op-ed and writing a 10-page research paper. All teams opted for the op-ed. When asked the primary reason for choosing the op-ed over the research paper, 87% of individual students responded that the sheer volume of a 10-page research report vs. a 700-word op-ed was the primary factor in their decision. If the intention is to provide students with real choices that promote self-initiation and autonomy, clearly communicating the true expected workload of all alternatives, along with all of the details for the task milestones, would be crucial.

Social Competition, Stress, and Grading
In previous implementations, the instructor observed that openly complementing successful first round publication in class or rewarding publications with better grades imposed unintended stress and anxiety on some students which were so far unsuccessful in publishing their op-eds. Since the op-ed writing project is considered a supplementary activity to the research project for promoting social responsibility and improving writing skills, we decided to keep the grade weight at 5% and not to discriminate published and unpublished teams in grading. The 5% grade was entirely dependent on the quality of the manuscript (according to the rubric), punctuality in the iterative review process and submission to newspapers, and professional conduct in interacting with the TA and editors.

Professional Benefits
Several students from the that successfully published their op-eds on a newspaper uploaded a link to the on-line article on their LinkedIn account. Considering the characteristic of LinkedIn as a professional social networking service, publishing their op-eds on a newspaper appears to be perceived as a significant achievement.
Students sign the op-ed submissions as "Univ. of …… students majoring in ……", and published op-eds bring publicity to the department, college, and university. Students not only take pride in educating the public on a timely business topic, but also provide exposure to the university through publishing op-eds. A department head at the college commented: "op-ed is a wonderful opportunity for students to practice their skills in writing publicly and is also very encouraging for them to speak their thoughts. It also is good for marketing the program."

CONCLUSION
This study presents the implementation of an op-ed writing project in an undergraduate SCM course, complementing a team research project on emergent SCM topics. We have found this project to be effective in teaching students how to write an op-ed, which entails learning a novel writing style for most students. We also found the project to be competent in teaching students how to publish their opinions in a newspaper, which includes the review process and interaction with journalists and newspaper editors.
We have found this project to be beneficial to our students by engaging them in the process of forming an opinion with strong grounding in the local/regional communities and participation in the public forum of policy debates and discussions.
Many students expressed, via survey and verbal comments, their excitement in learning that they, too, can have a voice in important global issues. By offering cutting edge business technology or contemporary global supply chain perspectives to the general public, this op-ed writing project serves as a social responsibility project for students as well. Finally, the project could also offer career benefits to the students by fortifying their critical thinking capabilities and collaborative skills, making them more valuable employees.
The op-ed writing project can be improved in various ways, including providing a broader choice of topics and teaming options. Utilizing the campus Writing Center (Kovach, Miley, & Ramos, 2012) or collaborating with a professional journalist (Poronnik & Moni, 2006) would help streamline the review process and improve the overall quality of student manuscripts. Further, op-ed writing may be offered to students as an optional project, allowing them to work in smaller teams.
Pay attention to the word count. exclusive count is for only the main body of the article. the second count is for the whole document, including title and bio notes.

The submission E-mail.
Your submission to the editor will be an e-mail, containing your cover letter and the manuscript.
Your cover letter will be the body of the e-mail. Attach your manuscript as attachment.
Should you choose to paste your op-ed in the e-mail, you should indicate by saying [Op-ed pasted and attached] at the end of the cover letter, instead of just [Op-ed attached].
It's up to you whether to just attach or paste and attach your manuscript.
[CRITICALLY IMPORTANT] BCC my email in the recipient of the e-mail. DO

NOT CC or include me as a main recipient.
This will be your submission proof to me.

The Submission Deadline.
Prepare according to the above instructions and send the e-mail to the editor by no later than 5PM on Thursday, April 18.

What to Do If Accepted.
In the case your manuscript is accepted for publication, forward the e-mail from the editor to me. Although publication is not a basis for your assessment, we will still celebrate.
May the Force be with you.
Your TA