The academic production on open educational resources in Portuguese

Open Educational Resources (OERs) have led to a new outlook on the circulation and production of educational content in print and digital forms. OERs have become an important element in the search for emerging and sustainable models for education, including initiatives such as open textbooks. In spite of its increasing popularity around the world, we still know little with regard to the academic production around OERs. This study presents a bibliometric analysis of the Portuguese-language production on OERs published until May 2015 and gathered from multiple databases and journals. The study presents a thematic analysis of the identified works and their association networks for institutions, and provides a critical outlook regarding academic production for OERs in Portuguese. The study aims both to contribute to the visibility of works published in less-prominent languages in academia such as Portuguese and to boost collaboration between authors who often discuss similar aspects of OERs but might be unaware of others working in similar areas of interest.

Los estudios sobre Recursos Educativos en Abierto (REA) proporcionan nuevas perspectivas sobre la circulación y producción de contenidos educativos impresos y digitales.Los REA son componentes importantes en la búsqueda de modelos educativos emergentes y sostenibles, como, por ejemplo, los libros didácticos abiertos.A pesar de su popularidad en todo el mundo, poco se sabe acerca de la producción académica en portugués con uso de REA.Este estudio tiene como objetivo describir y analizar la literatura académica en el idioma portugués sobre REA publicada hasta mayo de 2015, usando como fuente de consulta diversas bases de datos y revistas científicas que utilizan técnicas bibliométricas.El estudio analiza los trabajos identificados por temáticas y redes de instituciones, y se presenta una crítica sobre el estado del arte de la producción académica en idioma portugués con uso de REA.De esta forma, este estudio pretende contribuir al aumento de la visibilidad de la producción académica en lenguas menos prominentes como el portugués, y fomentar la integración y Openness has become a matter of concern for all levels of formal education through open science, open access, open data and the use and production of open source software (Aires, 2016).Amongst the many challenges raised by openness are issues of authorship and copyright (Caro & Lesko, 2014), new models for the purchase of textbooks, and teacher training.
Research on OER has grown substantially though little is known about the academic production related to this topic, particularly outside the Anglosphere.There is growing concern over the domination of rich, and particularly Englishspeaking countries in the production and dissemination of OER (Amiel & Santos, 2013).Beyond the substantial imbalance in the availability of open educational content in lesser-spoken languages, the same imbalance may be present in academic publishing.Arimoto and Barbosa (2012), Cobo (2013) and Zancanaro, Todesco, and Ramos (2015) mapped the scientific production related to OER, without focusing on Portuguese.Production in lesser-spoken/written languages might suffer from a lack of discoverability, particularly if traditional publication aggregation services are used, which favor English-speaking outlets.Moreover, researchers are constantly under pressure to publish in international journals particularly in English (Rego, 2014).
This study is part of a project that began in 2013 aimed at cataloguing the growing academic production related to OER in Portuguese.Since the first use of the OER at a UNESCO meeting in 2002, research around this concept has grown significantly.As evidence, the OER Knowledge Cloud sponsored by the UNESCO Chair in OER at Athabasca University (Canada) aimed at cataloguing OER production in English has over 1,000 items.
With these disparities in mind, this research project aims to shed light in an area of growing interest (OER) in a language with significant but smaller academic prominence (Portuguese).This work seeks to systematically present the production around OER by analyzing its main institutions, actors, outputs and themes, focusing on production in Portuguese worldwide.

METHODOLOGICAL PROCEDURES
Bibliometric research is characterized by the analysis of scientific production in order to identify indicators that can show the development of a particular area of knowledge (Bufrem & Prates, 2005).To this end, this bibliometric study was conducted in three phases, which expanded, and comprised of eight steps: 1) search, selection and cataloguing of individual works; 2) standardization and classification of works; 3) analysis and final document production.Each phase and its corresponding steps are shown in Figure 1.The explanation for each step is presented below.
Step 1 -Defining terms to be searched This search focused on works discussing OER, written in Portuguese.In line with the ethics of openness, our search was skewed towards open access publications and sources.For the purposes of this study, open access was defined in a comprehensive manner, including any resource that could be accessed freely without financial burden to the reader, not taking into account formats or legal permissions.
Step 2 -Defining where search will be made The research took place in two stages, following the criteria set out in Step 1.During stage 1, from 2013 to May 2015, the focus was on a select group of wellknown databases and periodicals based out of Brazil (Scielo, @Educa, National Repository for Theses and Dissertations/CAPES and the Brazilian Digital Computing Library), as papers published in conference proceedings and other relevant reports and academic work.In light of the fragmentation of academic production in various sites (different journals and conferences), it was deemed prudent to accept and incorporate these works as much as possible, in order to enhance the diversity of the Brazilian academic production.Finally, at a later moment, works from the Scientific Open Access Repository in Portugal were included.
In order to openly organize and publicize the identified works, an online database was built using Zotero (<https://www.zotero.org/groups/educao_aberta/items/collectionKey/ EFI4NXC4>).This base has always been available publicly, allowing anyone to make use of it and to suggest the addition of new works.
During Stage 2, conducted between 16 and 17 June 2015, the research project was expanded to include the Community of Portuguese Speaking Countries (CPLP): Angola, Brazil, Cape Verde, Guinea-Bissau, Equatorial Guinea, Mozambique, Portugal, São Tome and Principe, and East Timor.To this end, a search was conducted in registered journals in the Directory of Open Access Journals (DOAJ -<https:// doaj.org>), for journals classified under the subject Education and articles written in Portuguese within these journals.Also, searches were carried out in the repositories of the CPLP countries registered in the Registry of Open Access Repositories (ROAR -<http://roar.eprints.org>).Finally, searches were conducted in the library international aggregator WorldCat (<https://www.worldcat.org>), in the scientific research international aggregator OpenAIRE (<https://www.openaire.eu>)and in the Brazilian journals sponsored by the Coordination for the Improvement of Higher Level Personnel (CAPES), an aggregator of both closed and open journals.
Step 3 -Applying the criteria for work selection The search terms, as defined in Step 1, were used for consultations in repositories, aggregators, journals and proceedings, according to the locations defined in Step 2. Works had their title, abstract and keywords read to identify pertinence to the study.When in doubt, the full text was read.In addition, we sought books, book chapters, articles in journals and technical reports that were within the context of the study, but were not indexed in repositories or journals.
Step 4 -Including selected work in the software for bibliographic reference management After selecting the relevant papers for the study, those works that were not previously registered in the Zotero database were included with the relevant metadata. Step

-Standardizing and including other information than that available in the metadata
In order to expand the analysis of the identified papers, a database was created using Microsoft Access, to which the Zotero metadata was added.Also included, manually, were authorship information and place of publication.In addition, for each selected work, information was added regarding the institutional link, georeferencing of the institutions to which the authors were affiliated, and references (citations) used by the authors to compose their work.
Step 6 -Reading and classifying selected works In this step, the selected publications were thematically classified.After a first draft classification, researchers went through two cycles of consultation in order to adjust conflicting classifications and categories.
Step 7 -Analyzing data Once selected and standardized, queries were generated and visualizations defined (such as network views and maps).

Step 8 -Writing final report
Based on the analysis conducted in the previous steps, information was organized in an article format.

ANALYSIS OF THE RESULTS
General bibliometric search results are presented on Table 1, which shows the quantity of papers identified in Stage 1 of this research, as described above.Finally, Table 3 summarizes the collected data during both stages.Of the 107 selected scientific publications, 33 were articles published in journals, 29 were works published in conference proceedings, 16 were book chapters and two were books.A total of 17 dissertations, four theses and one postdoctoral report were found.In addition, two technical reports, an undergraduate course final project, a magazine article and a statement were found.
Table 4 presents the bibliographic data of the 107 selected works.Source: Research data * The term "sources" refers to the place in which the work was published, i.e., the institution/editor of origin.

Temporal trends
As shown in Figure 2, the first works were published in 2007, five years after the first seminar where the term "Open Educational Resources" was discussed at UNESCO (2002).There were two publications in 2007.The first one was the Cape Town Declaration (2007), which originated in an event sponsored by the Open Society Institute and the Shuttleworth Foundation in South Africa, which had as its purpose the promotion of openness in general, and open educational resources in particular.The document was the Portuguese translation (the original was written in English) and was included here as it is considered a milestone in the OER movement.The trend shows a very small increase over the years until reaching a peak in 2012, with a substantial production of 33 works.This was due to the launch of the book "Open Educational Resources: Collaborative practices and public policies", known in Brazil as the "OER Book" (11 works) published in print and online in Brazil; and the publication of "Open Educational Resources and Social Networks: Co-learning and professional development" (4 works) published online in the UK.During the same year, the first edition of the "Workshop on Open Educational Resources" (I WREA) was conducted, as part of the Brazilian Congress on Informatics in Education (CBIE), in which eight articles were published.Moreover, we identified five journal articles, three academic theses and one book.The year also coincided with the Global OER Congress held at UNESCO, which led to the Paris Declaration on OER.Based on the available data we consider 2012 a turning point for OER production in the Lusophone world, and particularly for Brazil.
Other conferences were held later on: six in 2013 and seven in 2014, in which works dealing with OER were published.The "III Luso-Brazilian Colloquium of Distance Education and E-learning", held in 2013 is noteworthy, with four papers; and the "III Brazilian Computer Congress on Education" held in 2014, which had two articles published.We take this as evidence of incipient discussions on OER within the regular tracks of conferences in both distance education and informatics and education.
Publications in scientific journals increased over time: there were five in 2012, eight in 2013, ten in 2014 and three in 2015 (up to when this study was conducted).The publication of scholarly articles in journals may suggest greater maturity in academic production, since these are usually the final or more mature products of research projects.

Main sources of publications
The 107 selected papers came from 64 different sources.The majority of works came from the "OER Book" (11), I WREA (7) and the RENOTE Journal (7).The first was the product of an open call financed by the Brazilian Internet Steering Committee (CGI).It is a compilation of theoretical reflections and reports, and five interviews with policy makers and practitioners on OER (interviews were not included as works in this study).The second was a workshop within a larger congress (CBIE), The event was organized by the Open Education Working Group (UNICAMP), as a follow up to an international event held the previous year in Logan, Utah (1st Symposium on OER: Issues for Localization and Globalization).Both can be considered outliers from the traditional flow of academic production (articles, dissertations and theses).
The first was a one-time publication; the second an OER-themed track that was part of a larger conference.The last source is a journal sponsored by Federal University of Rio Grande do Sul (Brazil) focused on information technology in education.
Table 5 presents the journals with the greatest number of published articles.From a total of 18, three events had the largest number of publications: the aforementioned I WREA; the "III Colloquium on Luso-Brazilian Distance Education and E-learning" held in Lisbon (December 2013), with four published works; and the "Third Brazilian Congress of Informatics in Education" (Dourados, MS -Brazil, November 2014), with two published works.

Main authors, institutions and countries
A total of 169 different authors were identified.Figure 3 shows the network with 55 institutions and affiliated researchers.Each dot represents an institution.Information regarding the authors' affiliation was extracted from the publications, and present day affiliations may be different from those presented below.One can notice substantial fragmentation.Authors tend to publish within their own institutions.There are only five clusters in Figure 3 ranging from 2-4 institutional connections.The larger network of association is detached and presented in Figure 4, which is comprised of 16 institutions.It is a small but decentralized network, which one can expect will grow into a more distributed form as collaborations grow (see, Baran, 1962).6 shows the institutions which have the greatest numbers of researchers.Brazil stands out with 77.5% of the authors, which one might expect given its size and population.Portugal has 14.2%, the UK has 5.9%, the US has 1.2%, and Germany and South Africa have 0.6%, completing the list of countries with authors publishing in Portuguese.14), Santa Catarina (13) and Ceará (11).In São Paulo, the institutions with the highest number of authors are: USP (10), Federal University of São Carlos (6), UNICAMP and PUC-SP (5).In other states (RS, PR, SC and CE) research occurs primarily in Federal Universities.In Portugal, there is a concentration of researchers mainly at the Open University, the Polytechnic Institute of Bragança and the University of Aveiro.In the UK, all researchers are affiliated with the Open University.

Main keywords and macro-themes of publications
In the 107 publications selected for this study, the authors used 206 different keywords to identify their productions.Not surprisingly, the terms OER (REA) and Open Educational Resources (Recursos Educacionais Abertos) were used 47 times in total, while the terms Distance Education (Educação a Distância) and Learning Objects (Objetos de Aprendizagem) were used 7 times.Open Education (Educação Aberta), Information and Communication Technologies (Tecnologias da Informação e Comunicação), Education (Educação) and Web 2.0 were used four times in total.The key word cloud (Figure 6) shows the main keywords used in the analyzed studies.The works were also classified in six macro-themes, as shown in Figure 7: material production (produção de materiais), policy (política), legal issues (questões jurídicas), open education/OER (Educação Aberta/REA), technology (tecnologia) and assessment/research (avaliações/pesquisas).

Main quoted references
Authors used 3,242 different references to produce their works.The five most mentioned studies are described in Table 7.The three most quoted works in the English language are general references on the subject.The work by Hylén (2006) is an OECD report that reviews OER.It conceptualizes the theme, clarifies institutions concerns with regard to engagement in providing open materials and presents challenges for the OER movement.Butcher (2011), was translated into Portuguese by the Information and Communication Technology Research Centre (CETIC.br).The author explores the concept of OER and argues in favor of the movement, discussing political issues and the use of open licenses.The "Forum on the Impact of Open Courseware for Higher Education in Developing Countries" is a reference to the event held in Paris in July 2002 at UNESCO in collaboration with the William and Flora Hewlett Foundation and the Western Cooperative for Educational Telecommunications (WCET).It is a reference commonly used to denote the first formal use of the term Open Educational Resources.
The "OER Book" (Santana, Rossini, & Pretto, 2012) was published by Casa da Cultura Digital (House of Digital Culture) and EDUFBA (Federal University of Bahia) with funding from CGI and the Open Society Foundations and support from the OER-Brazil Community.It presents a broad view of OER with articles discussing issues, dilemmas and initiatives with a focus on Brazil.In this book, the chapter "Educação aberta: Configurando ambientes, práticas e recursos educacionais" (Open Education: Configuring environments, practices and educational resources)

DISCUSSION
During the first step of this search, the only terms returning results were "Open Educational Resources" and "Open Resource".Other terms used to increase the search range were considered irrelevant.This might point to a welcome convergence around this terminology.
The word cloud shows the predominance of the term "objetos de aprendizagem" (LO; learning objects), which can be attributed to the fact that many researchers who worked (or still work) around LO also research and discuss OER.Despite major differences, many relevant issues in OER such as open licenses and formats, metadata standards and others, are legacies from discussions, limitations and outcomes of learning objects research (see, for example Parrish, 2004).
The most referenced Brazilian works are the "OER Book" and a book chapter from the same source.This provides evidence of the book as a milestone in creating awareness on the subject in Brazil (and perhaps abroad).Recognizing that the influence may be small, we can point to two works by authors linked to institutions in Portugal, who referenced the book.
Of the total works, 45% of the references used by the authors are works in Portuguese.In addition, 31.8% of the 107 works that comprise the corpus of this analysis are mentioned in the references of other articles, which represent 3.5% of all quoted works.In a joint analysis of such data, one can assume that there is not, yet, a "core" group of articles for OER in Portuguese.Given the recent spike in productivity around the theme (peak in 2012) it is expected that, in the coming years, with increased publication around OER, some works will form a cluster of seminal references.We see evidence of this in the analysis conducted by Zancanaro, Todesco and Ramos (2015) in which the most quoted works are "Giving Knowledge for Free: The Emergence of Open Educational Resources" published by the OECD in 2007; "Models for sustainable open educational resources" published in 2007 by Downes; and "A Review of the Open Educational Resources (OER) Movement: Achievements, Challenges, and New Opportunities" also published in 2007 by Atkins.All of these are also overviews which are often quoted in introductions on the concept of OER.These works, however, are used very little by authors writing in Portuguese.Works identified by Zancanaro, Todesco and Ramos (2015), combined with those most mentioned in this study, might form a consistent theoretical source for future works on OER.
Most of the analyzed works (46.7%) were written by a single author; 18.7% by two of them; 20.6% by three; 10.3% by four; 1.9% by five, and 1.9% by six authors.Data indicate that academic production is spread amongst a large number of authors and institutions.Despite the existence of institutions that have more salient production (Table 6), it cannot be concluded that there is a core group producing knowledge on OER.A small group of authors have a regular and significant production on the theme, which is not reflected in the representation of the institutions.This means that authors who produce more on the theme are not necessarily linked to the institutions with the largest number of affiliated authors.
Since the academic production on OER is still small as compared to the number of searched sites, we cannot identify the emergence of research groups or centers in different countries.20.6% of the selected works were dissertations, theses or postdoctoral reports.Of this total, 72.8% were published between 2012 and 2015.This shows academic interest in OER.79.4% of all identified studies were published in the last four years, so there is increased awareness and interest in the OER movement.
Unlike the English language, which has at least two open access journals (<http://irrodl.org>e <http://www.openpraxis.org>)and at least three major conferences focused on OER and open education (<http://conference.oeconsortium.org>, <http://openedconference.org> and <https://oer15.oerconf.org>),there are no equivalents for Portuguese.The closest one is the WREA, which held its second meeting during CBIE 2015 (and an upcoming III WREA in Spain).This is perhaps why publications on the subject are spread across different journals, with varying foci, particularly those related to technology and education.

CONCLUSION
This article aimed to identify the characteristics of research on OER published in Portuguese.The data used in this research was retrieved from 355 locations, through the selection of works relevant to the object of study (OER).Some technical considerations and limitations are discussed below.
The large number of locations that were searched reveals the lack of aggregators to provide reliable results.Both ROAR and DOAJ have a global search option.However, the results are different when the search is performed separately on each journal site/repository.Therefore, each search was conducted individually for each specified location, which required more time.In addition, in 36.5% of the analyzed studies, authors did not define keywords, reducing accuracy in the analysis, which make it more difficult to find these works through search engines.A further limitation is that this article focused its DOAJ search to journals under the subject Education and written in Portuguese.Other journals may publish works related to OER and would be outside of the scope of our search.Future works may include other categories under DOAJ, for a wider search.
Just over a third (31.8%) of the papers were not indexed in websites of journals or repositories, being included individually.This addition was necessary because of the nature of the production around OER.The inclusion stems from a more inclusive, rather than a seemingly impartial search, done only in indexed journals.This would result in a stricter but far less valid methodology.Still, important works may not have been included in the analysis despite the team's best efforts.With the growth of productivity on the subject through traditional academic means (such as graduate research), there might also be an increased output production in outlets that are formally valued by the academy (such as indexed journals).
Given the growing emphasis around the publication of articles in international journals (particularly in English), the academic production in Portuguese-speaking countries is likely under-represented.Future studies may investigate the production networks around OER and OE starting with authors and their institutions, regardless of the language of the work itself.
The analysis of the production on OER helps to map and identify relevant actors and issues for the OER community beyond the Anglophone sphere.Increasingly, researchers who are not native English speakers are put in a position to make tough choices with regard to the language they wish to publish their works in (including the ensuing extra time and costs).We must press that as a minority language Portuguese has limited visibility in academic publications and beyond.In this sense, we are quite aware that publishing works in minority languages is, in some instances, a political choice with detrimental effects to the authors.Authors might sacrifice international visibility and traditional academic rewards to make the results available to a wider, local, non-academic public or by prizing open journals published in Portuguese, for example.
As an example we can take the process of applying for a jobs at public, federal universities in Brazil, which use article publications as one aspect of candidate evaluation.A publication in a journal published in Brazil (usually in Portuguese) rated at the same level as a journal published outside of Brazil (using the Ministry of Education's Qualis system) will give the candidate less points --simply because it was published in Brazil.
As part of this effort we are now working with Athabasca University in their OER Knowledge Cloud (https://oerknowledgecloud.org) as it moves towards a multilingual database of the academic production on OER.Through this work, we aimed to increase the visibility of the academic production on OER in Portuguese; the same can be done for different areas of interest and research.Analyzing, promoting and connecting the academic production in a minority language is one way to relate these works to a wider discussion on an emerging field such as OER.

Figure 1 .
Figure 1.Research Methodology The second one was an article byDutra and Tarouco (2007)  that sought to give an overview of OER based on the then recent MIT OpenCourseWare experiment.

Figure 2 .
Figure 2. Publication type per year

Figure 3 .
Figure 3. Network of institutions based on author affiliation

Figure 4 .
Figure 4. Largest network of associations

Figure 5
Figure5shows researcher affiliation data on a map.The diameter of the circle indicates the number of authors.

Figure 7 .
Figure 7. Thematic analysis of publications is a theoretical overview on open education, reflecting on the role of open resources and open practices in light of institutionalized education.Other highly cited works include "Professores-autores em rede" (Networked teacher-authors) by Nelson Pretto with 11 references; "REA: O debate em política pública e as oportunidades para o mercado" (OER: The debate in public policy and market opportunities), by Carolina Rossini and Cristiana Gonzalez with 10 references; and "Educação Aberta: Histórico, práticas e o contexto dos recursos educacionais abertos" (Open Education: History, practices and the context of open educational resources) by Andreia Inamorato dos Santos, with six references.Finally, the work byDownes (2007) addresses the importance of OER, conceptualizes the terms resources and open, and explains the different business models in light of the sustainability of OER projects.

Table 1 .
Works identified in Stage 1

Table 2
presents the data obtained from the consultations during Stage 2 of the research conducted in CPLP repositories, in journals registered in DOAJ, WorldCat, OpenAIRE and the CAPES Journal Portal.

Table 2 .
Works identified during Stage 2

Table 4 .
Bibliographic data based on selected works

Table 5 .
Journals with the greatest number of published works

Table 6 .
Institutions with the greatest numbers of affiliated researchers

Table 7 .
Main references used in the analyzed works