Annika Werner, Onawa Lacewell, Andrea Volkens
Manifesto Coding Instructions (4th fully revised edition), May 2011
[pdf download (incl. category scheme, training test, and entry test)]
Introduction: CMP and the Purpose of this Handbook
This is the handbook for the Comparative Manifesto Project (CMP/MARPOR), which provides CMP coders with an introduction on how to apply the rules and definitions which constitute the data production process of the project. CMPs objective is to measure policy positions of all relevant parties competing in any democratic election in the post-World-War-II period for the following countries: OECD and EU members, Central and Eastern Europe, and (in the future) Latin America and South-East Asia. Analysing manifestos allows for measurement of party policy positions across countries and elections within a common framework. Manifestos are understood parties only authoritative policy statements and, therefore, as indicators of the parties policy preferences at a given point in time. For this reason, manifestos are chosen as the subject for quantitative content analysis.
This content analysis aims to discover party stances by quantifying their statements and messages to their electorate. A unified classification scheme with an accompanying set of rules was developed to make such statements comparable. This handbook provides coders with all of the relevant information, definitions, and sources needed to apply the coding scheme to their respective countries.
1. The Manifestos
Manifestos are chosen for the basis of this research from the various types of party-issued documents. The British label manifesto refers to what the rest of the world dubs election program, i.e., a text issued by political actors on the occasion of elections in order to raise internal and/or external support. For the purposes of this handbook, the term manifesto is defined as text published by a political party in order to compete for votes in national elections.
1.1 Which Parties?
The Comparative Manifesto Project aims to measure the policy preferences of each relevant party running in an election which is included in the data collection. Relevant parties are defined as those parties that win seats in their respective election. For Central and Eastern Europe, every party winning at least two seats is included in the data collection process. All coders hired by MARPOR will receive a list of parties for whom manifestos (or their substitutes) have to be coded. Coders are asked to propose corrections or amendments to the list if necessary.
1.2 Collection of Manifestos
In most cases, coders are asked to collect the manifestos for the elections they are to code.
While the definition of manifesto presented above may initially seem straightforward, manifestos can vary considerably across parties, elections, countries, and years. The title of a manifesto can differ considerably, from Election Program of Party X, Program, Platform or Action Intentions to statements such as We will make Australia prosper. Furthermore, in the event that parties provide more than one version of a manifesto, whether a long and a short version or several otherwise different versions of the manifesto, all versions need to be collected and sent to the supervisors. This is also true for instances where the party provides a machine-readable version (i.e. in doc or html format), a digital version (i.e. in pdf format), and/or a colour hard-copy including pictures. The ideal type of manifesto is the machine-readable format and should be collected whenever possible. For documentation reasons, however, the project also needs at least one fully formatted version of the manifesto, which is usually either in pdf or hard-copy format. If the former is not available, the latter is sufficient.
1.3 How to Find the Manifestos
The manifestos can often be gathered from the parties themselves and in particular their websites, special election newspapers of parties and/or regular newspapers. Furthermore, sources might be research and training institutes or publications, e.g. books with collections of programs, associated with the parties. If manifestos are not freely available, coders are asked to contact the party. In all cases, the ideal, machine-readable manifesto should be retrieved when possible.
In cases when no manifesto is available, this should to be reported back to the supervisors immediately. For instance, the only texts available may be newspaper summaries as a condensed form of the parties election pledges. Sometimes, only reports of party spokesmen about policy preferences and goals for the upcoming legislature may be available. In case only such documents are available, all available information needs to be collected very carefully under the supervision of the training supervisor.
1.4 Filling in the Manifesto Information Table for Every Coded Election
When providing the original manifestos to the supervisors, coders are asked to fill in one manifesto information table (see p. 13) for each election for which coders have collected manifestos. This information is crucial for the Manifesto Project team to be able to manage and collect all necessary secondary information.
2. Preparation: The Training
All coders must take part in training before they are allowed to start the production coding, i.e. coding real manifestos. Coders who have already coded actual elections need to retake the training every two years in case of uninterrupted production coding or whenever they have not engaged in production coding for more than six months.
2.1 Purpose of Training
Of central concern to this coding procedure is the comparability of results. Hence, in principle, every coder should make the same decisions concerning the unitising and coding of any given manifesto. To ensure comparability, the project has defined a set of coding rules that all coders should follow. The training assures that all coders have a sufficient understanding of the coding process, enabling them to create reliable and comparable data.
2.2 Process of Training
In order to ensure maximum reliability and comparability of data, the training process is based on intensive communication between the prospective coders and the supervisor within the MARPOR team. Within this process coders learn how to code party manifestos, in particular how to deal with the coding scheme and which rules apply under which circumstances. Thus, coders learn the rules and gain valuable initial insight into the coding process.
2.2.1 Communication with the Supervisor
The core of this training is the close communication between coders and the training supervisor. Within the MARPOR project team, there is always one scholar who acts as training and coding supervisor. This supervisor administers the training and helps coders with any problems during the training and the production coding phase. These problems might range from questions regarding rules or definitions to single sentences with which coders have problems. Whenever coders need any advice or clarification, they are urged to contact the supervisor. The training and coding supervisor is currently Annika Werner (werner@wzb.eu).
2.2.2 Reading the Handbook
The basis of the training process is this handbook. The instructions must be studied thoroughly and followed closely. It is important to highlight that it is not sufficient to simply look at the handbook once. Coders should read the handbook several times and try to commit all coding rules and definitions of categories to memory as much as possible. The more coders can memorize, the faster the production coding will be.
2.2.3 Taking the Tests and Receiving Feedback
Central features of the training process are the successful completion of two tests by every coder: one training test and one entry test. The training procedure is as follows:
1) After examining the handbook, coders code the first training test and send it to the supervisor via email.
2) The supervisor sends extensive feedback on the training test.
3) Coders code the second test—the entry test—and send it to the supervisor.
4) The supervisor decides on the basis of the quality of coding whether coders have sufficient understanding of the coding process to begin production coding. With the decision that coders should a) commence production coding or b) receive more training, the supervisor sends feedback on the entry test to coders.
If necessary, each test can be taken twice.
3. A Two-Step Process: Unitising and Coding – Basic Rules
After coders have sent the original manifestos to the supervisor and have successfully completed the training, the supervisor provides coders with electronically codable versions of the manifestos. Coders are then asked to code this version of the manifestos.
The central question of manifesto coding is: What message is the party trying to convey to voters? Which are the issues the party regards as important?
The decision-making process of coding is described in the following sections. This procedure comprises two steps: a) unitising (how many unique statements do parties make?) and b) coding (what kind of statements do parties make?).
3.1 Which Parts of a Manifesto Should Be Unitised and Coded?
Each textual part of a manifesto needs to be unitised and coded. Some parts of the manifesto, such as statistics, tables of content, and section headings should not be considered as text. Introductory remarks by party leaders should be similarly ignored. These parts, however, should not be deleted but instead kept for documentation purposes.
When preparing the manifesto for coding, the supervisor earmarks those parts of the manifesto that should be ignored. Coders are asked to code in accordance with this tagging procedure but also to check the decisions made by the supervisor. If coders doubt whether certain parts of the manifesto should be treated as text or not, they should seek immediate advice from the supervisor.
3.2 Unitising – Cutting Text into Quasi-Sentences
The coding unit is a quasi-sentence. One quasi-sentence contains exactly one statement or message. In many cases, parties make one statement per sentence, which results in one quasi-sentence equalling one full sentence. Therefore, the basic unitising rule is that one sentence is, at minimum, one quasi-sentence. There are, however, instances when one natural sentence contains more than one quasi-sentence, as discussed below.
3.2.1 When to Cut Sentences
Only if the natural sentence contains more than one unique argument should this sentence be split. There are two possibilities for unique arguments: 1) a sentence contains two statements that are totally unrelated; or 2) a sentence contains two statements that are related (e.g. they come from the same policy field) but address different aspects of a larger policy.
Clues to unique statements might be 1) semicolons; 2) the possibility to split up the sentence into a meaningful bullet point list; 3) general clues from codes. Regarding the third point, it is especially likely that the sentence includes two unique statements if a sentence contains codes from two or more domains (see Table 1, p. 6). An example would be:
We need to address our close ties with our neighbours (107) [F02F?] as well as the unique challenges facing small business owners in this time of economic hardship. (402)
3.2.2 When Not to Cut Sentences
There are many instances when sentences should not be split into quasi-sentences. A good rule of thumb is that one word is most likely not a quasi-sentence. It is crucial to know that examples, reasoning, explanations, etc. are not unique arguments and are therefore no separate quasi-sentences.
Coders should also be careful when unitising based on sentence operators such as commas, colons, hyphens, etc. Such operators might be, but are not always, indicators of a quasi-sentence. Operators do not indicate two quasi-sentences if they do not separate two unique statements. Examples for this case are:
The animal rights in our country must be improved; and we will do that. (501)
Our countrys budget must be put on solid footing again, no matter the costs. (416)
Coders should not split up a sentence just because they think they have discovered a code. For instance, the mere singling out of another country is not a unique argument and, therefore, a quasi-sentence. Only if the statement refers to a general or specific foreign policy goal should it be considered a separate quasi-sentence. Furthermore, references to policy areas such as education, agriculture, labour, and the environment should not automatically be separated simply because catch words such as schools, farmers, unions or environmentalists are mentioned. Again, the sentence should only be cut if it is a statement about the issue. Here is an example of a sentence that seems to contain several arguments at first glance but, on closer inspection, is revealed to have only one unique message:
We must force our unions to step back from their demands or their policies will result in the loss of thousands of jobs, closing of schools, and diminishing pensions. (702)
In this example, jobs, schools, and pensions are only instances outlining the negative impact of what will happen if the partys central demand (unions reducing their demands) is not met.
3.3 Bottom-Up Approach to Coding - Finding the Right Code for a Quasi-Sentence
3.3.1 The Categories
The Manifesto Project developed a category system whereby each quasi-sentence of every manifesto is coded into one, and only one, of 56 standard categories. The 56 categories are grouped into seven major policy areas and are designed to be comparable between parties, countries, elections, and across time.
Table 1: 56 Standard Policy Preferences in Seven Policy Domains
3.3.2 The Code Allocation
The following questions are central to the decision making of assigning codes to quasi-sentences: What are the statements of the party? Which policy positions does the party convey? In order to make this decision, coders need to make sure that they understand what the party says. Therefore, it is essential to read every singly quasi-sentence very carefully.
Often parties are very clear in their statements and candidly say what they seek: more of one thing, less of another. In this case, assigning codes is straightforward: coders identify the message and assign the corresponding category. When consulting the category scheme it is important to keep in mind that the categories definitions are not exhaustive. They are meant to give a general notion and some exemplary statements. The scopes of the categories are not constrained to the exact wording of the definition and it should be assigned to all issues that are related to the general idea conveyed.
There are, however, times when parties statements are not very clear and are more difficult to code. When facing such an ambiguous sentence, the coders should always first think about the meaning of the quasi-sentence and double-check the quasi-sentence with all codes in the category scheme. This helps assure that the quasi-sentence does not simply fall into one of the lesser used, rare categories.
In general, there are three possible factors which cause ambiguity: 1) Language is often simply ambiguous. Language is full of various styles, jargon, rhetorical meanings, colloquialisms, etc. Manifestos, therefore, often use language in manifold ways. 2) Quite often parties not only say what they want to achieve but also how they want to achieve it. Sometimes, coders will find both statements within one natural sentence and will have to decide how to handle this high density of information. 3) Many of the political issues included in manifestos are very complex and it is not possible to convey a clear message within one quasi-sentence. Parties often choose to build their arguments over several sentences, within a paragraph and/or sometimes even over the course of a whole chapter.
Coders need to keep these sources of ambiguity in mind in order to fully understand the message the party wishes to convey. The following section addressed ways for coders to handle ambiguous language and other problems during the course of coding.
1) Ambiguity of Language
a) Often, parties make policy statements by mentioning a negative aspect of an issue in order to highlight its importance for the party. Take, for example, the following:
Our countrys democracy does not work well enough anymore!
This sentence could be read and interpreted as a negative statement towards the countrys democratic processes. However, it is rather clear that the party is not making a statement against democracy itself. The actual message of this sentence is one of concern about and criticism of the current state of democracy. Therefore, this is a positive statement towards the ideal principle of democracy.
b) Furthermore, parties tend to use ambiguous or convoluted language to hide certain statements often deemed politically incorrect or inadmissible viewpoints. Coders should try to understand the message while at the same time trying not to read too much into the quasi-sentence.
2) Ambiguity of Quasi-Sentences because of Complexity – A Hierarchy of Context
When the quasi-sentence in itself does not convey an obvious message despite coders best effort to find one, several levels of context might be helpful to decide how to code a quasi-sentence. These levels are hierarchal. Coders should keep in mind that it is imperative to consider the context level closest to the quasi-sentence first and only move to the next level in case the closer one was not helpful.
The context levels are, in sequence from the quasi-sentence level upwards:
the rest of the sentence in case the quasi-sentence is only part of a natural sentence
the previous and the following sentences
the whole paragraph
the whole chapter or section
the whole manifesto
the political discourse concerning the issue in the country at the time of the election
3) Statements Containing Several Messages
Sometimes more than one code seems to apply to a quasi-sentence because the party wraps several statements up into one broad statement. Quite often, these statements come in the form of We want to reach A by doing B and C or We are doing B and C because we want to reach A. In principle, the grand rule of code the message applies. For these two examples, the message is that A is primarily important. B and C are simply means to achieve A. Goals usually take precedence over means when assigning codes. The following example claims that changing the constitution might serve the purpose of promoting animal rights. Since the constitution change is clearly only a tool, this sentence is not cut into two quasi-sentences and only the animal rights code applies.
To make sure that animal rights are universally recognised, we are going to add them to our constitution. (501)
However, there are instances where this logic does not apply. It might be possible that the party not only sends a message for A but also puts so much emphasis on B and C that B and C become messages in themselves. This is most apparent when the party states that B and C are the only means possible and there is an imperative to use them: We want A therefore we must employ B and C as the only feasible options. The following example is one where the means (leaving NATO and reducing the military) are such strong messages in themselves that they need to be coded separately from the goal (peace).
In order to achieve worldwide peace, (106) [F02F?] our country must leave NATO (105) [F02F?] and reduce the military to a minimum. (105)
4) Statements Containing No Message
There are instances when a sentence by itself does not make a statement. Often, the context helps in these cases and the rules mentioned above still apply. A special case is when sentences are used as a way to introduce or end an argument, or to connect two arguments. These introductory, terminal, or connecting sentences do not constitute meaningful statements themselves but are part of a continuous argument. Therefore, they should be coded in the same category as the corresponding argument or as the bulk of the paragraph in which they appear.
5) Proximity of Contradicting Codes
Finally, a note of general caution: it is possible to have positive and negative codes on the same issue right next to each other. Party manifestos often include contradictory statements. Coders should not try to assume hidden meanings in a quasi-sentence just to make sense of the sentences around it. Manifestos are not codes to be deciphered. Instead, coders should be careful to only code what is written. The following examples are seemingly contradictory statements in close proximity:
We will support our troops overseas, (104) [F02F?] while working to end the current war. (105)
Our constitution is a model for every truly democratic system (203) [F02F?] but we need to change it (204).
4. Specific Provisions - Rules to Keep in Mind
There are several rules for the process of code allocation that stem from decades of experience with manifesto coders. There are certain habits and behavioural patterns which all coders (and especially new coders) should try to avoid. Therefore, while the following rules might seem trivial, coders are asked to keep them in mind.
4.1 Rare Occasions: When to Use the 000 Category
Generally, coders should try to use a meaningful code (101 to 706) whenever possible.
However, there are instances when 000 is an applicable code. The instances are: 1) the statement is totally devoid of any meaning and cannot be coded within the context; 2) the statement refers to a policy position that is not included in the category scheme. This may be particularly true for modern issues or if the category scheme only includes codes in one direction (positive or negative) and the statement refers to the non-included direction. For instance: agriculture and environment are positive categories with no negative counterparts. If a statement can only be classified as Agriculture Negative it should be given a 000 code. All quasi-sentences treated as uncodable must be rechecked after the program has been coded in its entirety.
4.2 Catch-All Categories
303, 305, 408 and the 700-categories (except 703, see below) are meant to be catch-all categories for general policies that do not fit any specific coding category. They should always be avoided if a more specific policy category can be used. However, this does not mean that they are forbidden. Coders should double-check the usage of these categories to make sure they have not missed a specific policy.
4.3 Agriculture
When agricultural issues are mentioned, coders often have the choice between 703 Agriculture: Positive and another, often economic, category. In these instances, a special rule applies: If coders can choose between 703 or any other category, then 703 should be chosen.
However, this does not mean that the inclusion of the word farmer automatically makes the category 703. This category should only be assigned if the statement is actually about agriculture and farmers and is positive.
4.4 Sub-Categories
During prior phases of the Manifesto Project a sub-category system was developed for problematic statements not fitting within the main CMP coding scheme because of their country or area-specific bias. This was primarily the case for Central and Eastern European countries during the post-communist transition period. These sub-categories are still in existence but their usage should be avoided as much as possible.
To make sure that no sub-category is used unnecessarily, coders must contact the supervisor whenever they consider using a sub-category. In particular Central and Eastern European coders should contact the supervisor about this issue.
4.5 Background Knowledge vs. Personal Bias
All coders are expected to be citizens of the country they code. We use citizen coders because they benefit from their background knowledge of their country. However, background knowledge should not be confused with a coders personal characteristics, beliefs, and attitudes—all of which are potentially harmful to the comparability of coding.
Background knowledge is unique knowledge that only citizens of the country can have. It includes knowledge of the countrys history, social problems and cleavages, electoral issues, party system, and party ideology. Personal biases, on the other hand, are coders individual beliefs and attitudes concerning social and political issues, party ideologies, politicians and generally concerning what is right and wrong.
Coders should draw on background knowledge to help determine the code of ambiguous quasi-sentences only. However, coders should only do so if no other clues are available. In all cases, personal bias must be avoided! Such bias causes distortion. Coders should be especially careful when coding their most favourite and least favourite parties!
Furthermore, coders need to make sure that the statement is coded as it reads. If a party claims that their policy proposal has certain outcomes, this needs to be coded as it stands, even if coders think that these policies will lead to other or even opposing results. Again, the central focus of coding is to find out the policy positions and points of view of each party. Any personal judgements (of rightness or wrongness, whether a statement is realistic or sensible, etc.) need to be avoided. The following sentence is a good example:
We will increase the military expenditure to ensure peace in our region. (106)
This sentence might sound incorrect but, nevertheless, the party is conveying the message that they want to improve the regions prospect for peace (106).
5. In Case of Questions and Queries
A trouble-shooting system exists for cases of questions and queries. The contracting supervisor (currently Onawa Promise Lacewell) needs to be contacted for any issues concerning manifesto collection, training, and coding contracts. The database supervisor (currently Sven Regel) can be contacted for any questions and problems concerning the technical side of dealing with manifestos, especially on how to work with the online platform. The training and coding supervisor (currently Annika Werner) needs to be contacted for any issues regarding the coding, whether questions about coding rules, code definition or any other issues. Coders might also discuss the coding of special issues, in particular country specific issues. Furthermore, coders may translate single sentences or paragraphs to obtain advice on how to deal with them. All communication processes run trough the new online platform (http://manifesto-project.wzb.eu) or via email (manifesto-communication@wzb.eu). The coder is asked to indicate in the subject line to which supervisor the message is directed.
Category Scheme
Training Test
Country: Australia
Name of the Party/Alliance: National Country Party
Year: 1966
Title: We will grow, prosper (extracts)
Entry Test
Country: USA
Name of the Party/Alliance: The Democratic Party
Year: 2008
Title: Democratic National Platform (extracts)
