RESEARCH AND EVALUATION METHODS ASSIGNMENT

Assessment 1: Research Proposal Part 1

In the first assessment piece you will: identify a criminal justice intervention, develop a clear research question or questions, and provide a concise and focused literature review related to your research question. Specific criteria and a Template for this assessment can be found in the Course Profile. The following will help you think about your approach to this piece of assessment.

Assessment topics

For your assessment you will select one of the topics described below. Once chosen, this topic will be the basis for all three assessment items. You cannot change topic once your first assessment has been submitted, so choose wisely and think ahead to how you will address Assessment 2 and 3. The topics are:

  • Mental health courts
  • Prisoner employment and re-entry
  • Restorative justice conferencing

Based on the topic you choose you will then identify a criminal justice intervention related to this topic. Your task will then be to construct a project proposal where you will design an evaluation of this criminal justice intervention. Different components of this project proposal will be submitted in Assessments 1, 2, and 3. For more information see the Study Guide and the Course Profile.

What is a criminal justice intervention?

A criminal justice intervention is a program or policy that is put in place to address a particular crime or criminal justice problem, or to have some desired effect. Broadly speaking, examples of these effects or outcomes could be to: reduce crime, reduce recidivism, and improve victim outcomes. These are just examples, there are many other outcomes interventions may aim to affect.

Interventions may be programs or policies put in place by official government agencies of the criminal justice system – for example, police, courts or corrective services – however, these interventions can also be introduced and carried out by non-governmental groups, for example, youth service agencies, immigrant service agencies, mental health agencies, block watch or neighbourhood watch programs, women’s shelters, counselling services, legal aid programs.

An example of a criminal justice intervention topic is a drug court, an example of a specific intervention associated with this topic is the Drug Court of NSW, and an outcome associated with the implementation of the Drug Court of NSW may be reduced recidivism.

The criminal justice intervention you choose for your assessment will relate to one of the three topics repeated below:

  • Mental health courts
  • Prisoner employment and re-entrRestorative justice conferencing

In your assignment you will be required to clearly describe the criminal justice intervention to be evaluated in a concise and focussed manner.

Developing your literature review

Assessment 1 requires you to identify a criminal justice intervention, conceptualise and articulate a research question, and provide a concise, focused literature review related to the question you pose. Your research question will develop out of your literature review, although you may have some ideas in mind before you approach the literature. Taking the drug courts example, we might expect the Drug Court of NSW will help to reduce recidivism, however what does the literature say about drug courts, their role and their impact?

What is a literature review?

Once you have chosen your topic the next stage is to take a look at how others have thought about and researched this topic/intervention. The purpose of the literature review is to give you more information about how others have dealt the topic and to assist you to more specifically develop you research questions.

The exercise will help you to ‘ground’ your study within a larger framework of research so that you can better understand how to fine tune your research questions, how to select the appropriate methods for answering your research questions, and how to analyse the information that is produced. In this way your literature in Assessment 1, will also aid you to develop Assessments 2 and 3. In reviewing the relevant literature you will also gain information about why your research topic is important, and how it fits into a larger criminal justice policy or intervention framework.

In more detail a literature review:

  • is an account of what has been published in the area of your topic,
  • will help you define or redefine your topic and research question(s),
  • will help you to clarify your research strategy and design, and
  • will help you to identify why your topic is important and how it fits into a larger criminal justice policy or intervention framework.

The best way to help you to understand what a literature review is and why it is important is to being reading journal articles that include a literature review. You will do this during the course of your literature search.

Literature search

So how do you get connected to the literature in your chosen area? The most efficient and reliable way is to search library databases such as Web of Knowledge, SCOPUS, or Sociological Abstracts for example. A list of other criminal justice databases, including the ones listed above, can be found on the Griffith Library site.

Databases allow you to enter search terms related to your topic. As an example, if we were interested in studying the relationship between fear of crime and actual crime rates in urban areas, we could enter search terms such as “fear of crime” + “violent crime” + “neighbourhood” (or neighbourhood – to be inclusive of American spelling) to examine studies in the area. If you are interested in a topic that has a large amount of literature and you are having difficulty narrowing down the number of studies to look at, try to limit your search with more specific terms that could relate to the particular research method you are interested in, e.g., “evaluation”, or “survey” or “field research” or “longitudinal” designs. Another way to limit the number of studies to look at is to include search terms related to some of the concepts in your study. For example, you may only be interested in “juveniles” or “Australian prisons” or the “post-war era”.

You may also find Google Scholar to be a useful resource. If you find a journal article you are interested in you can then use Google Scholar to search all of the other papers that cite your journal article of interest. When using Google Scholar however, be aware of journal quality. While databases such as Sociological Abstracts contain only high-quality, peer-reviewed journals, Google Scholar is not restricted in this way.

At the masters level it is expected that you are familiar with literature searching methods. However if you do require help or additional advice please visit the library helpdesk and/or complete one of the self-help library research tutorials, or contact the course convenor e.g. https://www.griffith.edu.au/library/study.

What kind of studies should you include in your literature review?

Try and limit your research to high-quality, peer-reviewed journal articles, as well as authoritative books. Government reports may also be useful. For the literature review in this course it will not be necessary to be exhaustive about your review and to include all studies related to your topic. Instead, you should select the studies that are most relevant to your particular study, that assist in clarifying the topic and research questions, and that provide information about possible methods for answering your research questions.

Ideally, you will include three to five peer-reviewed empirical studies to discuss in detail, and draw on numerous other resources where appropriate. It is preferable that the empirical studies you review in detail, are not written by the same author and that each provides some additional information to your review. Studies should also be recent where possible (i.e. past 10 years). They must be empirical studies where the authors conduct research/analyse data. These studies should come from high-quality peer-reviewed journals.

Most of these empirical studies will include a review of the literature relating to the area of study. To help write your own literature review, take a look at how other authors have approached and organised their reviews.

What questions should you ask yourself for each article or study that you review?

  1. There a number of things to look for when you are reading the studies that you have located for your review. It helps to approach each article with the same set of questions. For example:
  2. How does the article relate to your research question?
  3. What methodology or research design have the authors used?
  4. What did they find?
  5. What population, sample, and units of study did they use?
  6. How do they conceptualise the terms proposed in their research questions?
  7. Do you see any problems or gaps in any of the components of their study design? For example:
    1. Have they properly operationalised their concepts?
    1. Could the research question have been approached more effectively from another perspective?
  8. Did they discuss whether and how their results ‘applied’ to future criminal justice interventions? Policy or program changes?

Remember to critically analyse these studies, rather than just reciting their findings.

When you are conducting and writing up your literature review, ask yourself…

  1. What is the problem, or research question that my literature review helps to define?
  2. What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)?
  3. How good was my information seeking? Has my search been wide enough to ensure I’ve found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I’ve used appropriate for the length of my paper?
  4. Have I critically analysed the literature I use? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
  5. Will the reader find my literature review relevant, appropriate, and useful?
  6. What are the “gaps” in knowledge and how can my research questions address these gaps?

For more information on what to include in your literature review please review the Assessment 1 Criteria available in the Course Profile.

Referencing

Use APA citation style: Please use the American Psychological Association (APA) citation style in your literature review and all of your assessment for this course. You will find information about how to properly cite your literature on the Griffith University Library website: https://www.griffith.edu.au/library/study. You must include a reference list at the end of each of your three assessment pieces as well as using in-text citations. Using in-text citations are necessary to avoid plagiarism as well as to demonstrate you have done the work. Most of the time you should be putting information in your own words, quotes can be used on occasion as long as they are referenced appropriately and put in quotation marks.

Developing a research question

Once you have chosen your topic and identified a relevant criminal justice intervention you will begin to develop your research questions. These research questions will relate to the expected outcomes of the intervention. The process of developing a researchable question is iterative or a back and forth process. You will begin with a general topic in mind, consult the available literature, refine your question, and consult the literature again. After thinking about your topic and consulting the literature, your next task is to focus your ideas and to translate them into a more concrete and specific research question.

There are a number of things to consider in formulating a research question:

It must be an empirical question.

In other words, the answer to your question must be ‘discoverable’ through observation or experience.

Examples of these types of questions could include:

  1. Is ticketing for public nuisance offending a cost-effective alternative for the police?
  2. What are the effects of long-term incarceration on offenders?
  3. Are vulnerable groups over-represented as offenders arrested by the police?
  4. Is youth violent crime increasing?

Although there may be ways we could improve these questions, in each case, you can envisage real-world information that can help to inform the answers to these questions, and you can imagine strategies for collecting that information.

It must not be a question of value, philosophy or politics.

These are not empirical questions, and the answers to these questions cannot be ‘discovered’ without specific criteria to identify the “right” answer.

Examples of these questions could include:

  1. Is rehabilitation a better objective for sentencing than punishment?
  2. What should we do about pornography?
  3. Is incarceration an appropriate sentence for the mentally-ill?

Note that research can be done in these areas, but empirical research cannot be carried out based on the present form of these questions.

In each case, the questions are asking the researcher to make a value judgment using words like “appropriate,” “better,” and “should.” In order to empirically research such questions, it is necessary to properly define or specify the criteria for these words of ‘value’.

As an example, we may be interested in exploring incarceration as a sentence for the mentally-ill. We could improve the question “Is incarceration an appropriate sentence for the mentally-ill?” by defining the word “appropriate”. While there are many possible ways to define appropriateness, one possibility is to specify that appropriateness is defined by a reduced risk to re- offend.

In this case, the question could be transformed as follows: “Is the risk to re-offend lower for mentally-ill offenders sentenced to incarceration or to alternative sentences?”

You must know and define your evaluation criteria.

Researchable questions need to have specific evaluation criteria so that you can tell whether you’ve answered the question. As an example let’s take the question “Is ticketing for public nuisance offending a cost-effective alternative for the police?”. The intention is to evaluate the “cost- effectiveness” of replacing arrests with tickets (similar to a traffic ticket) for low-level offending such as public nuisance. If you were actually tasked with researching this question, you would need to more clearly define what you are evaluating since “cost-effectiveness” can have a variety of meanings. Does cost-effective refer strictly to savings in police time? To a monetary cost- savings for the police? To social benefits for the community? Or to some alternative measure of cost-effectiveness? The question could become: “In cases of public nuisance do police save time by issuing tickets rather than having to make arrests?”

To the extent possible, your question should be specific and limited in scope.

Overly broad research questions are difficult to design research strategies to assess. It is necessary to define the limits of your research, either in the scope of time you will study, the geographic region, or the characteristics of your units of study. As an example, it is possible to narrow the scope of the question “In cases of public nuisance do police save time by issuing tickets rather than having to make arrests?” further by specifying the region of study and the time period of study. The question could become: “Based on the results of a 12-month trial of ticketing for public nuisance in Queensland, Australia, did police save time by issuing tickets rather than having to make arrests?”

You need to consider the feasibility of the question.

It’s never too early to begin to think about your possible research design or how feasible it is to carry the research out (or indeed the relevant project management and ethical implications of your study). In some cases good research questions cannot be examined for a range of reasons including:

  1. the inability to access data;
  2. the sensitivity of the subject matter;
  3. prohibitive costs;
  4. privacy concerns
  5. sample size

Summary: key points to consider in developing researchable questions

  1. Question should be expressed in researchable terms:
    1. an “empirical” question, not a question of value
    1. related to some empirical reality (i.e., must be some sort of evidence that can be consulted or observed), and
    1. should have specific evaluation criteria so that you can tell whether you’ve answered the question.
  2. Researchable questions should also be:
    1. specific
    1. limited in scope.
  3. Researchable questions should be feasible.
    1. It must be possible to design a study to answer your question.
    1. In developing your research question start to think about things like possible types of research methods, e.g., survey versus observation, units of analysis, cross-sectional versus longitudinal designs, population and sample, availability of data,  methods, analysis, ethics, budget etc.

Hypotheses

In addition to your research question(s) many of you will also be able to develop a set of hypotheses for your study. Hypotheses are statements about the results that you expect from your study. These statements are based on previous research and theory. For example, based on previous research or theory, you may expect that your intervention will have either a positive effect or a negative effect on the outcome you are interested in. Often, more than one hypothesis will flow from your research question.

The purpose of hypotheses in research studies is to: guide the direction of your study, identify the facts relevant to the study, help you to select your research design, and ultimately, help you to provide a framework in which the results of your study will be given.