Abstract

The domain of automated journalism is experiencing swift expansion and holds the potential to transform both the production and consumption of news. This study examines the news readers' perceptions of news articles generated by bots and human journalists. Using an exploratory research design, this qualitative study selected participants based on their familiarity with automated news content. Potential participants were approached via survey, and semi-structured interviews were conducted with 12 university students. Thematic analysis was used to analyze the data. The findings highlight news structuring, and engagement, and authorship accountability when it comes to automated journalism.

Introduction

The field of automated journalism is rapidly growing and has the potential to revolutionize news production and consumption (Jung et al., 2017). Automated Journalism is “the process of using software or algorithms to automatically generate news stories without human intervention—after the initial programming of the algorithm” (Graefe, 2016). Thousands of news articles are generated each day by algorithms at major news organizations with little assistance from human reporters where news articles are written in various domains such as sports, finance, weather, and education (Diakopoulos, 2015).

A large number of companies around the world are developing automated software to generate news. For instance, the Associated Press started using Automated Insights’ Wordsmith to produce sports and business reports in 2014, and as a result, there was a ten-fold increase in published stories because of automation (Associated Press, 2019, (Associated Press, n.d.). In one sense, scholars and practitioners recognize this technology's potential for improving news quality, but journalists are concerned that it may also result in the loss of job opportunities in newsrooms (Graefe, 2016).

Definition of key terms

For this literature review, key terms are defined as follows.

Algorithm: An algorithm is a step-by-step process for carrying out a task or solving a problem. It is a clear-cut set of instructions that accepts a value or group of values as input and outputs a corresponding value or set of values (Kim et al., 2016).

Large language model (LLM): LLM refers to a specific type of language model trained on large amounts of data for conversational purposes.

Natural language generation (NLG): NLG refers to the process of generating human-like text or narratives using computational algorithms. It involves converting structured data or information into coherent and understandable natural language (Dong et al., 2022).

Automated Journalism: Also known as algorithmic journalism (Dörr, 2015) or robot journalism (Carlson, 2014), refers to the use of artificial intelligence (AI) and natural language generation (NLG) technologies to automatically generate news articles and reports without the direct involvement of human journalists.

Research question

As observed in the review above there are fewer studies on the attitudes, perceptions, and experiences of readers towards automated journalism from an HCI perspective. Based on previous studies (Oh et al., 2020) (Perttu Hämäläinen et al., 2023), this study will focus on the general public perception of news articles generated by bots.

How do news readers perceive news articles generated by bots compared to those written by human reporters?

To address the research question, a qualitative study was conducted. A qualitative study allowed the researcher to gain a deeper understanding of individuals’ thoughts and opinions, and participants' perspectives on Automated Journalism. The data source for this study was acquired through interviews and surveys. The questionnaire was designed and developed based on previous research (Perttu Hämäläinen et al., 2023) (Graefe et al., 2016) (Wu, 2019) which conducted similar studies to evaluate readers' perceptions of automated journalism. Moreover, all the data-gathering methods for this study followed ethical procedures.

Data on generating news articles

For this study, two news articles were presented to understand the general publics' perceptions of human-reported versus algorithmically generated news articles. Three relevant and current news topics (weather, education, international news) were compared with one news article generated by the research using an AI tool (such as Google Bard). The other news article was taken from a published article produced by a human reporter from news websites such as BBC or the Guardian.

Human-reported data

For human-reported data,

• International news article was taken from the BBC website

• Weather news article was taken from the BBC website

• Educational news article was taken from The Guardian website

Detailed information available upon request. None of the articles were edited or shortened except for removing images. This was done to assure external validity (Graefe et al., 2016).

Bot-generated data

For bot-generated data, Google Bard was used to generate news on similar topics. Prompts generated for each article were as follows:

• International news: The prompt used was “Create a news article on northeast India Manipur”

• Weather news: The prompt used was “Create a news article on UK weather June 2023”

• Educational news: The prompt used “Create a news article on UK Universities guidelines on generative AI”

All three bot-generated news articles were generated on 3rd July 2023 and were sent for an ethical approval review. Detailed information on the bot-generated articles available upon request. None of the articles in the bot-generated version were edited or shortened, nor were they expanded with additional prompts.

Data collection

Detailed data collection information available upon request.

Participants were recruited from a population of university students through an online survey sent out via Google Form in July-August 2023. The survey questionnaire is available upon request. The survey was in English language and aimed to collect information on

• Consent to participate in the study

• Participants' news consumption habits, and

• Demographic information

Out of the total 15 responses received on the survey, only 13 participants showed their interest in being interviewed and 2 participants refused to participate in the interview. Basic demographic information such as age, gender, and level of education were collected to describe the sample of people who participated in this study and to have a better understanding of the participant's background. A brief summary of the participant's demographic information is presented in Table 1. Furthermore, all the collected information was anonymized by assigning each participant with a unique identification code. This was done to protect confidentiality and anonymity.