Part II
Additional Style Considerations
Formal
Style Requirements
Writing
Style
Part III
Implications of the
Research Report
PART I: THE INDIVIDUAL COMPONENTS OF THE ARTICLE
The burden of effective communication falls on the researcher. The report must provide information in a readily understandable manner. Thus, the report should avoid burdening the reader with unnecessarily abstract language, jargon, convoluted sentences, and a dense style. Research reports should crisply convey the nature and results of research to an intended audience and still make lively (or at least interesting) reading.
Research reports, however, are not constructed like detective stories. They do not build to a climax. Rather, they prepare the reader for what is to come, then present the research, and finally summarize what has been said. The Title and Abstract prepare the reader by providing an orientation. Statements of the Perspective and Problem inform the reader of the research intent. The Methods and Findings sections tell what the researcher has to report. And the Conclusion summarizes what has already been said. Research reports are often organized in this manner: title, abstract, perspective, problem, methods, findings, and conclusions.
Each of these sections will now be reviewed in detail.
The title of a report should be clear and concise. It communicates to potential readers what information they can expect to find. Titles which promise more than they deliver deceive the reader and justifiably are severely criticized.
A title can be created either before or after the research report is written. Creating a title before the report is written may guide in the writing of the report. Since a research project may investigate several problems simultaneously, selecting a title in advance of writing the report assists in separating the relevant from the irrelevant. On the other hand, a title can also be prepared afterward. This increases the likelihood that it will correspond to the report's content. Frequently, researchers create a tentative title before writing the report and then revise it afterward. In this way, they gain the advantages of both approaches.
Titles vary with the medium of the report and the intended audience. An examination of reports written on topics similar to yours and with a similar audience will provide clues as to proper title construction and format. Basically, there are two types of audiences: academic and nonacademic. Titles vary with the type of audience you are trying to catch.
If your audience is academic, you should create a title which generalizes the nature of the problem you investigated and the findings you obtained. This will assist potential readers in locating your research among that of thousands of others which are competing for their attention. It will also assist prospective readers in bridging the connection between your research and others on related topics.
If your audience is nonacademic, you should avoid abstract professional
language both in the title and elsewhere in the report. There are
two reasons for this. First, such language is more likely to serve
as an obstacle than an aid to communication to readers outside the profession.
Second, nonacademic readers are more likely to be interested in the specific
problem and findings of the research than in its implications for theory
or broader problems.
Contents
II. Abstract (Do not use in the 209 paper----use in longer papers submitted for publication)
Literally hundreds of articles and reports to private and public agencies are competing for reader attention each month. Readers use titles as a first step in locating research of possible interest. Next, they often review the abstract or executive summary which follows the title.
If a research report is being prepared as a journal article it usually contains a brief abstract of fifty to one hundred words. If the report is being prepared for a private or public agency, it usually contains an executive summary consisting of five or more typewritten pages. THE OBJECTIVES OF THE ABSTRACT AND EXECUTIVE SUMMARY ARE IDENTICAL. BOTH DESCRIBE THE RESEARCH PROBLEM, THE METHODS USED, AND THE FINDINGS OBTAINED. However, they vary as to length and detail.
On the basis of the abstract or executive summary. Readers will determine
if the reported research deserves their closer attention. Consequently,
the abstract or summary should pithily describe the essentials of the research.
The choice of language should be governed by it communicative power.
In academic reports, the careful selection of professional terms permits
effective communication in terse language. In nonacademic reports,
professional terms should be avoided when possible. In this case,
the language of intended readers should be used in order to communicate
effectively.
Contents
III. Perspective
There are three required report components--introduction, review of the literature, and specific discussion of the research problem--which are formatted differently from one journal to another. These three components are sometimes all included in a single section labeled "Introduction" and at other times there are each in individually labeled sections. Herein, the introduction and review of the literature are included in this section labeled "Perspective", whereas the specific discussion of the research problem is in the "Problem" section which follows.
The introduction repeats with slightly greater elaboration the essential contents of the "Abstract"--what is the problem or question investigated, what methods were used, and what are the findings. Some have suggested considering your entire report as an hour glass. It begins with broad general statements, progressively narrows down to the specifics of your particular study, and then broadens out again to more general considerations in the concluding section. Subsections may also be viewed as resembling an hour glass. In the introduction you may start with general statements about the topic and then narrow to the specific aspect to be investigated in the study. For example, you may begin with general observations about voting behavior that anyone can identify with and then move to the specific scholarly theory. Only for an exclusively academic audience do you begin by plunging into the jargon ladened heart of some theory. The ideal introduction lets everyone know what is to come in the following pages and can be understood by both specialist and nonspecialist. Think of two different people reading your report. One is a highly trained specialist who understands all the jargon and sophisticated methods. The other is not familiar with either the substance or the methods. Writing to both individuals in a single report is a real challenge.
Contents
For academic audiences, a research write-up normally begins with a review of the literature. This is designed to locate the problem within a research tradition. For readers unfamiliar with the literature, the review serves an educational function. For those familiar with past literature, it provides an overview on how the researcher perceives the issues. For all readers, the review legitimates the research; that is, it serves to justify why the problem was important and worth investigating. For example, your review might demonstrate (1) how there is a hole in the literature for no one has yet to investigated the relationship between variable "x" and the dependent variable "y", or (2) how the inconsistent empirical findings in the tests of a hypothesis can and need to be resolved, or (3) that there is reason to believe that a well documented relationship is weakening, etc., etc.
A review is necessarily selective. It cannot encompass the hundreds of past reports, articles, and books which have been published on the problem you may have investigated. There are three criteria often used in selecting which literature to incorporate in a review. First, it is important to select only those works which significantly bear on the research problem. Those which are of marginal relevance can be ignored. For example, if your study is focusing on why people do or do not vote, the literature focusing on how voters cast their ballots can be ignored. Typically, you would, through a citation, tell the reader where he/she could find a review of the literature on how people cast their ballots. Your efforts, however, will focus on only the literature dealing directly with whether or not people vote.
Second, you must include all the directly relevant research or a representative
sample of this literature. You need not go into great detail about
each article or book. In fact all but the most important articles
are often aggregated and then treated as a group. For example, you
might list
(see the style section for an example) all those articles that
have found religious preference to be related to whether or not people
vote. If the directly relevant literature is extensive, you
need to organize it in a systematic fashion and then cite representative
studies of each major organizational point. For example, you
might organize your literature review of whether or not people vote
on the basis of what independent variables others have already
investigated. Then as you discuss each independent variable
you can cite representative studies. Although a review is selective,
it should not seek to bias the reader. Thus, selected citations
should be representative and they should be reviewed fairly and impartially.
Third, while most reviews usually incorporate both something old and something new, they make a special effort to report the results of prior systematic empirical research on the topic. Reviews include references to classical writings of persons of unusual importance to the discipline, and references to the most recent significant writings. Equally important with what scholars "think" about the topic, is reviewing the empirical research on the problem. What parts of the problem have been investigated and which have not? How good have the prior research efforts been? Are the results consistent or inconsistent from study to study?
If your report is written for a nonacademic audience, an extensive literature review may be unnecessary. If such a review is desired, it often cites only the current and relevant works. Nonacademic audiences are seldom interested in the linkage between your work and previous writings or in the historical context within which your research falls. They are interested in recent work which directly bears on the research problem you investigated. Consequently, if a review of literature is included in a report for a nonacademic audience, it is important that a direct link between the literature and the problem you are reporting on be specified.
For more detailed treatments of writing the literature
review see
Writing a Literature
Review,
Using the Literature,
What Is a
Literature Review
Contents
If you review has been thorough, a statement of the problem should
flow naturally from the previous discussion. Readers should
be given a succinct, but clear idea of the research problem as you
perceived and structured it. Many of your own problems will
be solved if you have a direct, clear and specific statement of the
problem to be investigated. If your project involves
testing hypotheses, they will be formally stated in this section.
You must formally define the dependent, independent, and control
variables. In addition, it must be clear to the reader why
the predicted relationships are expected and why you are controlling
for the included control variables. A good review of the
literature will have provided a firm foundation for these tasks.
For example, your review of the literature might have reported that
there are several well established causes of voting. But for
some reason(s) (e.g., hunch, logical reasoning, deduction from some
theory, partial results in some previous study, etc.) you believe
yet another variable is related to whether or not people vote.
But, before your research project testing whether this additional
variable is related to voting is taken seriously, you must control
for the well know causes covered in the review of the literature.
Thus, your statement of the problem must indicate how the new variable
you are testing fits into the relationships that have already been
established by previous research. Is the general causal model
additive or interactive, direct or indirect? By carefully specifying
what all the relationships are thought to be, you will greatly simplify
your task of how to control in the next section.
Contents
V. Method
In this, as in other sections of the report, you must be selective. It is impossible and undesirable to inform readers about every detailed step in the research process. Ideally you report enough of the details so that the research project can be duplicated by someone reading the report. However, the method section is one area that is often shortened in journal articles in order to pare a manuscript down to the length demanded by editors. There are many research procedures, statistical models, observational techniques, etc. that are so well known in a field that you only need to briefly discuss them. You must mention any unusual steps you took, but you need not include all the details of standard practices in the field.
There is consensus that the methods section should cover: the specific research design, techniques of variable measurement, sample, data sources and collection procedures, and analysis procedures.
Remember, the broad conception of research design includes, among other elements, all of the topics included here in the methods section whereas the narrow conception of research design refers only to logical models of proof. What is the logical model being employed? Is it an experimental design-- randomized two group post-test only, randomized three group pre and post test, randomized four group factorial? Is design expost facto-------time series, panel, correlation cross-sectional? A knowledgeable reader will see the strengths and weaknesses of the design you are using. However, other readers can use some guidance as to both strengths and weaknesses and why you are using the design--the why may simply be that it is the only possible design for the problem given the resources at hand.
Contents
B. Techniques of Variable Measurement
In essence you must clearly lay out your operational definitions. Exactly how was each of the variables in the study measured. For example, you explain if a Likert or Guttman scale was employed, what procedures and decision rules were used for grouping subjects into categories, which theoretical and operational definitions were used, and so on. Researchers must discuss any known biases or compromises which were made with respect to specific variable measurements. The reader must know the strengths and weaknesses of the operational definitions for this may be very important in interpreting statistical results.
If you are using well know operational definitions you need not go into great detail, but can simply refer the reader to a more complete treatment to be found in the original source. If you are using a new operational definition, you must present it completely.
Contents
C. Theoretical Population and Sample
Your unit of analysis and population may be stated or implied in your discussion of the problem. If the population has not already been identified, you must specify both it and that upon which the research is to be conducted. Is your theoretical population (1) all sovereign states, (2) the 50 US states, (3) the 435 members of the US House of Representatives, (4) all voting age adults in the US, (5) etc.? Is the study to be conducted on the entire population or a portion of the population? If you are just using a portion of the population, how was it chosen? If the sample is not a probability sample, why not?
If you are doing a secondary analysis of data gathered by someone else, you need not go into great detail about how a sample was selected. For example, if you are using one of the well know National Election Study data sets, it is sufficient to note that the sample is representative of US adults and refer the reader to National Election Study documents that detail exactly how the sample was selected.
Contents
D. Data Sources and Methods of Data Collection
While sample selection is obviously part of the data collection process, the methods of data collection section must provide readers with specific information about the rest of the data-collection techniques. For instance, whether the project involved a survey or some other method of observation, whether the research used questionnaires or interviews, what conditions existed when data were collected, whether certain dimensions of the research design had to be compromised, whether the research experienced data loss from its beginnings to completion, and so on. A detailed statement of data-collection techniques is essential if readers are to develop confidence in the study.
Researchers must disclose known defects as well as positive aspects of the data-collection procedures used. This attention to strengths and liabilities applies to your collection of original data or to your use of secondary sources (e.g., statistical abstracts, government documents). Most of the time secondary sources (e.g., a statistical abstract) will contain all the details about how the information was collected and manipulated. You need not report all of this information, but be sure to include a citation so readers can find it if they so desire.
Contents
Your description of analysis techniques provides readers with
information about specific techniques you intend to use. For
example, your analysis may involve using percentages, ratios, cross-tabulations,
measures of association, or chi-squares. If you intend to use a complicated
or unfamiliar technique of analysis, readers should be informed as
to literature sources so they may consult these for clarification.
This permits them to review or learn more about the techniques you
will employ.
Contents
Some authors, particularly in long articles, prefer to break the results section into two parts. The first, usually labeled results, presents the empirical results with minimum discussion. Then a second section, usually labeled discussion, discusses the results in terms of the original research question and what other meanings and interpretations he/she chooses to include. Both of these topics are covered herein.
The Findings section of the report provides what most readers are reading to find out--the results of the research. This section therefore must provide a focused description of what has been found with regard to the questions and expectations raised by the title, the statement of the problem, and earlier sections of the report. Hence, it must be both selective and careful in how the information is presented.
The Findings section should not contain all that has been found in the course of the research. Many interesting observations and tables must be excluded because they are not sufficiently relevant to the thrust of the report. Some such findings, however, may be included in footnotes. Remember, you do not want to overload the reader with so many details that he/she looses focus on the main issue(s).
A number of conventions and rules facilitate this goal. In presentations featuring tables, the first of these usually gives an overview of the data (note again the hour glass format). Often this consists of a tabular presentation of the characteristics of the population or a sample if sampling was used. It provides data in a form that allows the reader to develop an orientation for the analysis which is to follow. Subsequent tables focus on the results of the analysis itself, presenting more specific aspects of the research.
For all tables it is essential that readers be provided complete information on what is in the table. Not all readers will draw the same inferences from data or they may wish to analyze it in a different way. Consequently, it is essential that sufficient information be provided so that readers may make their own calculations.
Furthermore, tables and figures should conform to the conventional rules for presenting data. For example, if you a presenting a contingency table (1) the independent variable will be on the top, (2) the dependent variable will be on the side, (3) the percentages will be calculated down the columns, (4) if the absolute values are not included in the cells of the table, they must be in the marginals, (5) the sample size must be included, (6) what the two variables are and where the information came from must be clear from the title, variable names, and/or table footnotes.
Many other conventions apply, each specific to particular forms of research. Examining previous studies similar in method and appealing to a kindred audience will give an indication of what your readership expects from the report you are preparing. For the novice writer especially, it is usually best to provide information according to the most familiar and tested format.
As was implicitly hinted at above, the general rule in reporting your findings is to give the forest first and then the trees. This is true of the results section as a whole: Begin with the central findings, and then move to more peripheral ones. It is also true within subsections: State the basic finding first, and then elaborate or qualify it as necessary. Beginning with one of your most central results proceed as follows.
1. Remind us of the conceptual question you are asking (For example, "It will be recalled that the men are expected to be more expressive than the women." or "We turn first to the question: Are the men or the women more expressive?") Note that this is a conceptual statement of the question.
2. Remind us of the actual operation performed or the actual behavior measured. (For example, "Do the men produce more tears during the showing of the film than the women?") Note that this is an operational statement of the question.
3. Tell us the answer immediately and in English. "The answer is yes." or "As Table I reveals, men do, in fact, cry more profusely than the women."
4. Now speak to us in numbers--present your statistics. (The reader not versed in the methods and statistics presented can skip to the next finding. Those who want details can read through the specifics.) "Thus the men in all four conditions produced an average of 14 cc more tears than the women, F (1,112) = 5.79, p <.025."
5. Now you may elaborate or qualify the overall conclusion if necessary. "Only in the Father-Watching condition did the men fail to produce more tears than the women, but a specific test of this effect failed to reach significance, t = 1.58, p <.12."
6. You should not take elaboration to mean a license to unleash an avalanche of only statistics and test results without indicating their meaning in terms of substantive results. You have failed if a knowledgeable reader must ask why a particular statistics is being reported.
7. Every "major" set of findings should be accompanied by a table, graph, or figure showing the relevant data (unless the entire set of findings can be stated in one or two numbers). The basic rule here is that readers should be able to grasp your "major" findings either by reading the text or by looking at the figures and tables.
8. End each section of the results with a summary of where things stand. "Thus, except for the Father Watching condition, which will be discussed later, the hypothesis that men cry more than women in response to visually-depicted grief appears to receive strong support."
9. Lead into the next section of the results with a smooth transition sentence. "Men may thus be more expressive than women in the domain of negative emotion, but can we assume that they are also more willing and able to express positive emotions? Table 3 shows that we cannot..."
By structuring the results section in this way, by moving from forest to trees, by announcing each result clearly in prose before wading into numbers and statistics, and by summarizing frequently, you permit the reader to decide just how much detail he or she wants to pursue at each juncture and to skip ahead to the next main point whenever that seem desirable.
After you have demonstrated that your quantitative results are statistically reliable, it is often useful to become more informal and to describe the behavior of particular individuals in your study. The point is not to prove something, but to all richness to your findings, to share with the readers the "feel" of the behavior.
One final note. Researchers are obligated in the Findings section
to report data which contradict as well as support any hypotheses
or ideas which were investigated. Only by reporting such information
can scientific knowledge grow. Contradictory evidence may lead to
important insights for future investigation. The researcher
must also report data that are inconclusive. The Findings section
therefore will often contain a number of highly tentative and qualified
findings, reflecting the efforts of the researcher to neither over
interpret nor under interpret the meaningfulness of the data.
Contents
This is usually a brief section designed to serve three functions.
First, it refocuses the reader's attention on the main issues and
findings and away from the myriad details which have been just presented.
Second, when appropriate, it qualifies some of the findings by stressing
methodological limitations or alternative interpretations of the
research. If, for example, the empirical results are directly contrary
to the original hypothesis, the author will summarize the reasons
why this may have been the case. Third, it may suggest directions
for future research based upon the experience of the researcher.
Such suggestions should not be broad statements that more research
would be useful (which is invariably the case), but specific statements
about the types and strategies of research that appear most promising.
Contents
VIII. Notes (optional)
If your are using the American Psychological Association's preferred
style of including citations in the text (see the style sections
for examples), the only footnotes you will have will be explanatory
and they go at the end in a section usually labeled "Notes."
Explanatory notes contain more detailed conceptual issues, data,
analysis issues, or interesting findings that, if included in the
text, would obscure the main findings.
All books and articles cited in the text of a research report
are listed at the end of the report under the heading "References."
Often if the list of data sources they are placed in distinct groups.
Follow your style manual for the proper format of each entry.
X. Appendix (optional)
An appendix is a good place to include very detailed information
which is inappropriate for the body of the report. For example,
you might include (1) the specific details of a new operational definition--the
questionnaire, (2) a more detailed description of a new, at least
to the field, statistical technique, (3) the raw data, if there are
not too many cases, collected in the study, etc. While journal
editors, because of space limitations, are not inclined to allow
an appendix, their inclusion in undergraduate reports is a good practice.
Contents
PART II: ADDITIONAL STYLE CONSIDERATIONS
The basic components of a research paper and their usual order have been presented herein. There are, however, many variations on formal style--types of headings and subheadings, footnotes vs endnotes, works cited vs bibliography, bibliographic entry form, etc. If the journal or your instructor has not assigned a style, pick one (e.g., American Psychological Association, Publication Manual) and follow it consistently. Increasingly the style of citation advocated by the American Psychological Association has become popular. In this style your citations are in the text and any footnotes are explanatory only. For example, "According to Festinger (1957), people find cognitive dissonance uncomfortable. Not everyone, however, agrees with this conclusion (e.g., Abelson, 1968; Bem, 1967; Kermit, 1979). Nevertheless, direct evidence for internal discomfort has actually been demonstrated in at least one study (Zanna, Freud, & Theophrastus, 1977)."
Contents
The previous pages have set forth numerous style conventions, but these are far from exhaustive. Unless you have three to four published novels to your credit you can benefit from owning and using any of the handbooks on good writing. What follows are a few style comments that are particularly relevant to empirical research reports.
Accuracy and Clarity--
The overriding criteria for good scientific writing are accuracy and clarity. If your report is interesting and written with flair and style, fine. But this is still a subsidiary virtue. First strive for accuracy and clarity.
Work from an Outline--
Even though the standardized format described here will go a long way toward organizing your report, you will be able to produce a more coherent report with a minimum of rewriting if you first organize the main points in outline form, examine the logic of the sequence, check to see if important points are omitted or misplaced, and so forth.
Write Simply, Use Examples, Use Friends as Reviewers---
It should be possible for a nonprofessional to read your report and comprehend what you did and why--even if he or she know nothing about statistics, experimental design, or the substantive area of your research problem. This is achieved by writing simply, with a minimum of jargon, and using frequent examples to illustrate and introduce technical concepts. The more abstract the subject matter, the more you need examples to tie it back to the reader's own experience and previous level of knowledge.
Read over your own writing, trying to take the viewpoint of an intelligent but nonprofessional reader. It is not easy. And because it is not easy, you should use your friends as reviewers, especially those who are unfamiliar with the subject matter area. If they find something unclear, do not argue with them or attempt to clarify the problem verbally. If they have read carefully and conscientiously, they are always right. Their suggestions for correcting the unclarities may be wrong, even dumb. But as unclarity detectors, readers are never wrong.
Person and Voice---
In the past, scientific writing employed the third person, passive voice almost exclusively ("The experiment was designed by the authors to test the hypothesis that..."). This was often dull and clumsy and is no longer the norm. It is now permissible to use the first person and desirable to use the active voice. Do not refer to yourself as "the author" or "the investigator." Do not refer to yourself as "we" unless there really are two or more authors. You may refer to yourself as "I" as long as you do it sparingly,; constant use of the first person tends to distract the reader from the subject matter, and it is best to remain in the background. Leave the reader in the background, too. Do not say, "The reader will find it hard to believe that..."
Tense--
Use the past tense when reporting the previous research of others ("Rummel reported..."), how you conducted your study ("Telephone interviews were conducted..."), and specific past behaviors of your subjects or participants. Use the present tense for results currently in from of the reader (Table I clearly shows the relationship between....), and for conclusions that are more general than the specific results.
Be Willing to Restructure-- The best writers rewrite nearly every
sentence in the course of polishing their successive drafts.
A often used strategy is to get the first draft out even if it is
very rough. Then go back section by section polishing and improving
each portion. Be prepared for major alterations, even repeating
part of the study, if a logical flaw is discovered.
Contents
PART III: IMPLICATIONS OF A RESEARCH REPORT
It is automatically assumed that you will attempt to provide an honest and valid view of your research. However, within the framework of honesty, which constitutes the first canon of scientific reporting, there are many other variables which will consciously and subconsciously influence your report.
First, as hinted, a research report is selective. It reports only information you wanted your readers to read. Scientific norms dictate that you report certain information such as that regarding data sources and data collection; but having discharged this responsibility, you have considerable flexibility in deciding what other information to include. By selecting some information and ignoring some other, your report will necessarily contain some bias. The latter may not be personally recognized and therefore may influence your report in unknown ways. Discussion with colleagues and self- reflection may curb biased reporting.
Second, the document is personal. It is testimony to your competence as a researcher. Since it may influence your self-esteem or even your career mobility, there may be a natural tendency to emphasize the strengths of the research
while underplaying its weaknesses. Again, this may not be a conscious decision. You may honestly believe that some of the weaknesses of the research are unimportant. However, this conclusion should be reached by your colleagues, and consequently it is important to include a discussion of strategies and procedures employed in sufficient detail so that readers can judge the merits of the research.
Third, the document is persuasive. Although the proper stance of a scientist is unflinching skepticism, it is difficult to be skeptical of one's own work. Therefore, research is usually presented in terms of arguments designed to support hypotheses rather than disprove them. This tendency to argue for the support of hypotheses may result in overlooking contradictory evidence. To curb this tendency, researchers should focus on negative as well as positive evidence and provide sufficient data for readers to draw their own conclusions.
Fourth, the report represents reconstructed history. It does not usually reflect how you engaged in all phases of the research nor does it necessarily reflect the order in which you engaged in them. To aid readers in comprehending your research, you will have portrayed it in a logical fashion, beginning with the perspective and ending with the findings and conclusions. This may provide a false image of your interaction with your data. The logical and stylized presentation is necessary for effective communication of research; however, in the process the empirical reality of false starts, dead ends, and premature decisions may be lost.
In sum, research reporting--despite its impersonal and logical
format-cannot be divorced from its social setting. As writer and
reader these elements must be considered in judging the scientific
merits of reported research.
Contents
PART IV: OUTLINE FOR CRITIQUING A RESEARCH ARTICLE
The following outline from Walizer and Wienir, Research Methods and Analysis, is what to do in critiquing an article. This is useful for not only evaluating the literature, but also after your draft is complete turn the questions on your own work. Are there obvious weakness that need to be corrected?
Title of Article
Authors(s)
Name of
Journal, Volume No., Year, Pages
I. General Concerns for Problem Statement
1. What is the problem or what is the general
issue at stake?
II. Conceptualization
1. What are the major concepts?
2. How clear are the conceptual definitions?
III. Hypothesis and Theory
1. Are there clearly stated hypotheses? If
yes, what? If not, are they implied?
2. Is the form, extent, and precision of expected
relationships made explicit?
3. Are the hypotheses designed to add to knowledge?
Do they utilize previous science?
4. Are any of the hypotheses tautological,
true by definition, or composed of
overlapping
concepts?
5. Is the temporal order between concepts
well established?
6. Does the review of literature allow for
meaningful interpretation of results?
IV Operationalization and Measurement
1. Is the transition from concepts to indicators
adequately explained?
2. Is the reliability and validity of indicators
discussed? If yes, how adequately?
If no, does this present
a major problem?
3. Are the indicators and/or manipulations
specified and adequately defended?
4. Are the cutting points and categories reasonable?
V. Design
1. What is the nature of the research design?
2. Are the strengths and weaknesses of the
design acknowledged?
3. Could the design have been easily modified
to improve the study? If yes, how?
4. What are the problems of the design in
terms of confounding influences and in
terms of generalization?
5. Is the research population clearly defined?
6. Is the population appropriate for the question
and/or hypothesis?
7. Is the type of sample specified?
8. Is the sample adequate for the kinds of
questions and/or hypotheses the study
attempted to investigate?
Why? Why not?
VI. Data Gathering and Analysis
1. Are the data appropriate for the
study?
2. Do the scatterplots, percentage tables,
or other statistics reflect the questions asked
in
the article?
3. How well are the problems of statistical
inference covered?
4. How clearly are the three characteristics
of a relationship (form, extent, and precision)
described where appropriate?
5. Does the analysis appropriately reflect
the control variable(s)?
6. Are the control variables properly located
in the research design?
7. Are there obvious control variables which
should have been included in order to fully
understand the main
relationship(s)? If yes, what are they?
8. Are the conclusions consistent with the
tables and/or figures and statistics presented?
9. Are there alternative conclusions that
are consistent with the analysis?
VII. Overall Comments
1. What is your overall impression of the
adequacy of the study for exploring the general
issue or problem raised
by the research?
2. What is your evaluation of the balance
between the limitations (weaknesses) of the
research and
its importance and contribution?
Contents
Return to PLS 209 Home Page