Tidying up Old Papers: Article separation in Chinese students’ journals

This experiment is part of a larger research project that seeks to re-examine the impact of liumei xuesheng (Chinese students returned from America) on modern China and the building of Sino-American relations during a foundational century (mid 19th-mid 20th centuries), based on a new bunch of digitized sources and new methodologies (e.g., natural language processing, historical network analysis, topic modeling). Crucial to this project is the creation of a textbase of students’ journals running over forty-five years (1906-1949) to enable full-text search and complex queries in vast bilingual (English/Chinese) collections (Table 1). Such a textbase will further allow historians to build question-driven corpora and to conduct quantitative and qualitative text analysis across multiplie articles, issues, and collections.

Table 1. Major collections of liumei’s journals before 1949. 

These journals are rich sources for documenting the liumei‘s discursives practices aimed at legitimizing their position in society, the social clubs, associations and activities they established to organize themselves as a self-conscious group, the local and global contexts in which they evolved in China and the United States, the political and social events that most concerned them, their literary writings and philosophical essays, their daily life (job announcements, classified advertisements), their sources of funding and their networks of patronages (commercial advertisements). Two of these journals have been the object of specific studies (Bieler, 2003; Chen, 2013), but in as much as scholars had to rely on print materials and manual methods of selection, they were not in a position to take fully advantage of the extent and depth of the journals’ contents.

From print to full text

These journals have benefited from the worldwide, massive digitization of historical press materials which began two decades ago with Google/HathiTrust. Digitization opens new opportunities for research but at the same time it raises important challenges. The scanning of journals is usually done at the page level, which raises well-known problems for analyzing documents with complex layout such as journals and newspapers. Their pages often include multiple columns containing both textual and visual contents. In the current state of the art, it is therefore impossible to make refined queries at the level of single articles or to filter (out) specific items such as advertisements, tables, illustrations captions. To enable the identification of unique articles with clearly defined boundaries beneath the page level, and to refine the classification of documents types, it is necessary to manually annotate samples of documents to create a ground truth. This ground truth can then be used to train a model to automatically detect and classify text contents in entire collections of (non-annotated) journals.

In this preliminary experiment, we used the Chinese Students’ Monthly (CSM) as a proof of concept to create our first ground truth. The CSM has been entirely digitized by Brill. Part of the collection is also available for free on Internet Archive. The CSM provides an ideal test case because it presents an intermediate structure between newspapers (which include multiple columns and advertisements) and  scientific journals (which include specific features such as tables, figures and footnotes). Furthermore, the layout of the journal evolved significantly during its period of existence, which allows us to cover a wide range of possible layouts. We identified three main  periods corresponding to three major types of layout, with an increasing degree of complexity:

1. Early years (1906-WWI): single column, mostly text;
2. 1920s: single column, textual and visual contents;
3. Late years (early 1930s): two columns, text and visual contents.

Coco Annotator Interface

• Paragraph: the most basic unit of text.
• Article: the most basic semantic unit (from a thematic or authorial point of view). An article can be composed of one or multiple paragraphs. In the latter case, we draw boxes for each paragraph and then we draw a larger overlapping “Article” box to comprehend all the paragraph contain in the article.
• Paragraph bottom: in case when the paragraph is interrupted at the end of the page.
• Article continued: in case when the article runs over two or more pages. In case the article runs over non-continous pages, we also tag “Continued p.” with a specific box.
• Heading : main title (first level) usually title of major section.
• Sub-heading : second level of title (sub-section or title of article)
• Sub-subheading : in case when there are more than two levels of titles.
• Author : named author of an article
• Advertisement
• Illustration
• Caption (of illustration)
• Tabular (tables, lists)
• Other : metadata and other contents we choose to discard in substantial text analysis but which we are still interested to retrieve for metadata analysis (e.g., information on editorial board, subscription rates…)

Fig.2. Labels used for annotating the CSM

Cécile Armand

Ancienne élève de l'Ecole Normale Supérieure de Lyon (2006), agrégée d'histoire (2009), docteure en histoire (2017), mes travaux portent sur la publicité, l'histoire urbaine, les migrations intellectuelles entre la Chine et les Etats-Unis et les méthodes computationnelles appliquées à la recherche historique. Mes recherches ont reçu le soutien des fondations Chiang Ching-kuo, Coca-Cola, Andrew Mellon (Program DHAsia, Stanford), de l'Agence Nationale de la Recherche (Access ERC, ENS-Lyon) et du Conseil Européen de la Recherche (projet ENP-China, Aix-Marseille).

More Posts - Website

Follow Me: