Over the past few months, the JSSWR staff has been working to create guidelines for qualitative research manuscripts. Our intent is to help authors increase the rigor of their reporting of qualitative studies, and thereby, to strengthen manuscripts submitted to the Journal.

To help us further refine the guidelines and to contribute to what we hope will be a useful tool for social work researchers, we are interested in receiving comments and feedback on our work to date.

To leave a comment, you must enter your name and e-mail address (comment box at bottom of page), but only your name will be posted with your comment. Comments are monitored, and all comments will be posted unless the author requests otherwise or the comment is deemed inappropriate.

The sections and subsections are numbered to facilitate referring to specific points, such as ” I think 2.3 in Lit Rev should include…” or ” Does 3.1 include assent of minors?”

In advance, we thank you for your time and sharing your opinion.
Mark W. Fraser – Editor
Diane Wyant – Managing Editor
Shiyou Wu – Editorial Assistant

Post Script: For those interested in the process used to develop these guidelines, a preliminary report is posted here:Preliminary Report -Developing JSSWR Guidelines for Qualitative Research

 

Comments Period: July 1, 2015 to August 31, 2015

** Comments Period is now closed **


Guidelines for Qualitative Research Manuscripts

JSSWR is committed to publishing high-quality research. We ask authors of quantitative manuscripts to follow standardized reporting guidelines (e.g., TREND, CONSORT-SPI, or PRISMA guidelines). However, to our knowledge, no standardized reporting guidelines with broad support exist for qualitative research manuscripts. This gap is due, in part, to the nature of qualitative research, which encompasses a wide variety of methods and perspectives, making it difficult to provide a single, comprehensive checklist. However, to strengthen manuscripts reporting qualitative research, we recommend authors consider the following.

 1. Introduction 

1.1  Describe the background and research question(s), including the study setting or context and the significance in terms of practice or policy

1.2  Link your research questions and/or specific aims to important social or health problems

1.3 Briefly discuss the compatibility of your research questions with qualitative inquiry

2. Literature review

2.1  Review recent research findings on the topic

2.2  Briefly discuss alternative theoretical perspectives (depending on relevance)

2.3  Note limitations in methods in addressing complexities or nuances of the problem

2.4  Summarize both the strengths and limitations of previous studies

3. Method
The Method section of the manuscript should provide sufficient detail to fully inform readers of the processes and procedures used to carry out the research. This section is key to making the research process transparent to the reader, and should include the following sections:

3.1 IRB Statement

All manuscripts reporting studies on human participants should include a statement regarding review and approval (or waiver) by the relevant institutional review board and/or research ethics committee. The informed consent procedures (i.e., oral or written) should be described in the Method section. In addition, describe the steps taken ensure participant confidentiality or anonymity, and the procedures used to ensure data safety. If pseudonyms are used to protect participants’ anonymity, be sure to note this protection. To protect the identities of study participants, avoid lengthy description of study sample or site, omitting details not essential to understanding the method or findings.

3.2 Methodology

Identify your research perspective or tradition (e.g., biography/narrative research, ethnography, grounded theory, phenomenology). Briefly describe any theoretical lenses and sensitizing concepts used in the study. Explain the rationale for selecting this method of inquiry and how your methods meet your stated aims.

3.3 Recruitment and Sampling

Provide a detailed account of how participants were recruited and engaged for the research study. Describe the sampling method (e.g., purposive, convenience, snowball), which should flow from the guiding methodology. For example, if using phenomenology, a sample of eight to 10 participants might be appropriate, whereas a larger sample size would be expected with grounded theory.

3.4 Data Collection

Describe the data collection methods (e.g., interviewing, observation, document review) in sufficient detail to inform the reader about the potential richness of the data (e.g., more than one interview per participant, extended time spent doing observation). Include information regarding who collected the data and their training or background. Highlight data triangulation and note limitations related to using a single form of data collection at one time point. If unique or unusual methods are used for data collection, describe them in detail and explain the rationale for their selection.

3.5 Analysis: Provide details of the steps taken in data analysis, and the process by which you arrived at the conclusions. Full description is important for the reader to get a sense of how themes were established and to strengthen the trustworthiness of the findings.

3.5.1  If coding was used, be sure to state how many people were involved in co-coding. If more than one coder was used, explain how consistency – if viewed as important – was established between coders and how conflicts were resolved. In addition, describe the steps taken to increase the credibility, transferability, auditability, confirmability, or other goodness metrics related to the findings. These can include peer debriefing, audit trail, negative case analysis, prolonged engagement, triangulation of data, and member checking.

3.5.2  If used, the type of data analytic software should be mentioned as helping to organize and retrieve data.

4. Results or Findings

4.1  The Results section should clearly present the themes abstracted from the analyses and — unless the study is purely descriptive — this presentation should move beyond the level of description-only. Authors can enhance the Results section by noting complexities within the results and, when possible, pointing out unexpected or surprising findings.

4.2  Although the use of participant quotations enriches qualitative manuscripts, do not overuse quotations within the Results section. Link quotes to the findings and your interpretations.

4.3  Including a graphic or schematic chart can be helpful in guiding the reader through the Results or Findings section and illustrating how the themes fit together. Color graphics can be included for articles posted online, and JSSWR can include a limited number of color graphics in each print issue.

5. Discussion

5.1  Synthesize the findings and link them to the research questions

5.2  Note whether and how findings fit within or advance the literature

5.3  Describe the ways in which the findings contribute to the knowledge bases for practice or policy

5.4  Explain the implications of the findings for practice, research, and/or public policy

5.5  Discuss the strengths and limitations of the methods in the context of your research perspective. For example, limitations related to sampling should be identified within a qualitative framework. The sample size or lack of depth in the data might limit transferability; however, a lack of generalizability is not a limitation in qualitative research

 

JSSWR recognizes that qualitative research is diverse and that all guidelines—no matter how expertly crafted—will apply differentially. Overall, authors should aim to include sufficient detail and description to make the research process transparent to the reader. Strengths and limitations should be assessed within a qualitative framework of credibility, transferability, auditability, and confirmability (see Lietz & Zayas, 2010; Lincoln & Guba, 1985; Padgett, 2008).

Acknowledgements

Many people contributed their time, talent, and expertise to the development of these guidelines. We thank each for their contribution and for their interest in promoting the rigor of qualitative research. Special thanks to the many people who reviewed the guidelines before we posted them — especially Laura Abrams, Cynthia Lietz, and Deborah Padgett, each of whom gave extensive feedback on earlier drafts of the guidelines. Special thanks also to Bruce Thyer, editor of Research on Social Work Practice (RSWP). Bruce shared a draft of the guidelines RSWP has recently adopted.

During this comments period, we hope you’ll help us think through guidelines for qualitative submissions for JSSWR. Because qualitative methods are so broad, these guidelines will not be as specific as those developed, for example, by CONSORT–Social and Psychological Interventions. We hope, however, to find some common language that will promote the credibility, transferability, auditability, and confirmability of all qualitative methods.

Mark Fraser

53 Responses to “PROPOSED Author Guidelines for Qualitative Research Manuscripts Submitted to the Journal of the Society for Social Work and Research (JSSWR)”

  1. Joan Pennell

    I think that the guidelines overall can assist authors using certain types of qualitative methods such as content analysis and constant comparative method. Other forms of qualitative inquiry might appear to be excluded such as those which would not use the terminology of “analysis” and instead use “interpretation” or “deconstruction.” I realize that the paragraph at the end acknowledges other forms of “qualitative research.” My concern is that some authors may be discouraged from submitting their work.

  2. Lia Levin

    Just some quick points regarding the introduction:
    1.2: I would suggest using the term “pertinent” rather than important. Importance implies that some social or health problems are unimportant, and thus not worth researching. A great deal of qualitative approaches would not necessarily accept such an assumption as relevant.
    1.3: This seems reasonable, although I would paraphrase this statement. As in guidelines for quantitative research this would probably not be required, such a guideline once again positions qualitative inquiry as residual by nature. As authors are already encouraged to justify various elements of the methodology they employed and its concordance with the research questions, it would seem redundant to also address the very choice of qualitative research in the introduction.
    Thanks for this!

  3. Leila Wood

    Overall, I appreciated the guidelines and think they are inclusive and sensitive to the nature of qualitative inquiry. However, the information about how much data (in this case, participant quotes) to provide needs more information. This depends on method. Furthermore, participant quotes are data and contribute to rigor and transparency. While quantitative researchers may limit the amount of statistical analysis to the research question/hypothesis, we would not ask them to not display pertinent numbers. Such as with qualitative research- too few quotes create limitations in rigor, trustworthiness, and authenticity. We want to see how the themes are supported by data, and a substantial amount of data may be needed to do this. I would favor guidelines that supported the use of enough data need to support themes, without the guidance to avoid overuse unless you can provide examples of overuse. As written, I am concerned that it encourages limited use of data. Too many articles are published without enough participant voice to support themes. Overall, I found these guidelines to be very thoughtful and also could serve as a great teaching tool.

  4. Sheryl Zimmerman

    JSSWR Staff: Articulating these guidelines promotes the rigor of qualitative inquiry and related published work; thank you for your efforts in that regard.

    I have one suggestion, related to 1.3, which reads “Briefly discuss the compatibility of your research questions with qualitative inquiry.” I suggest that the compatibility be addressed not only in the context of qualitative inquiry, but also in reference to the state of quantitative knowledge related to the topic area.

  5. Andrzej Klimczuk

    1 I do not see the info about the citation or references style [if this is important in this Guidelines]
    2 maybe at some point it will be good to underlinde that the papers need proper balance between professional language and jargon

  6. Heather Goltz

    I believe that the proposed guidelines are comprehensive and reasonable in terms of what you’re asking authors to submit for review. I also agree with the issues my colleagues have raised in previous posts in terms of how the proposed guidelines may be extended or clarified. One point of clarification relates to acceptable word counts or page lengths for qualitative manuscripts under the new guidelines. Will you begin allowing longer manuscript to accommodate the additional guideline-specific information that will need to be incorporated into qualitative manuscripts?

  7. Joanne Riebschleger

    Hello folks,

    I would be more clear in the section of quotes. The problem is usually not over-quoting but under-quoting depth due to space limitation. Much of the qualitative journal space gets used in very detailed methods sections. There is even less room for findings that are made up of text. Overly annotated or deleted text quotations can make qualitative findings look thin and peripheral. They can lose the rich voices of the study participants. This is an important research consideration. There must be more space (even one extra page makes a HUGE difference) for qualitative data. This would increase my submissions to the journal. I have had other well known journals okay this in advance when I asked for a bit more space in regular (not special) issues.

    You did not address the “numbers” of participants issue – in other words, numbers of people reporting on particular findings and themes. This is NOT usually appropriate to qualitative data (See Wolcott’s work). You should accept respondent data breadth described with the words most, some, a few, several, nearly all, etc. If ALL of the research participants are screened for an easy to measure construct, then numbers might be appropriate. For example, “All participants were asked if they worry about …..” – in this case, the forced choice format of the question might work for numbers (6 of

    and say that if the authors are confident numbers contribute to explaining the data they will be allowed.

    Creswell (2007) recommends that researchers use at least two methods of increasing data trustworthiness. This might provide a benchmark for the journal.
    The most critical omission herein is to say that the the journal seeks to avoid any bias toward qualitative research. There can be, as many of you know, the idea that if something can be reported in numbers it is more reliable than something reported in words. The underlying assumption by many reviewers is that quantitative research is bias free. Clearly that is not the case. Therefore, how will you train reviewers to understand and apply these standards. For example, are you using reviewers for manuscripts who are experienced in using qualitative data methods? Some doctoral programs require no qualitative method education and most do not require more than one course.

    I note that medicine has begun to include more qualitative methods as an important part of generating scientific knowledge development using many applied avenues. I am pleased that SSWR is paying attention to this.

  8. Joanne Riebschleger

    Here are my ideas:
    1. The primary issue is UNDER-quoting, not OVER-quoting. This is a major revision need in the document. The text excerpts ARE the data. When the data are reduced to a few words or multiple quotes are not allowed, the rich voice of the participants are lost. Further, the data looks thin and peripheral. The page limits are key since so much detail is required for qualitative work methods. This allows even LESS space to highlight findings. I routinely ask for, and am often allowed, extra space in some very good journals (regular issues). Even ONE extra page helps a good deal as qualitative data cannot be easily reduced to a few words in a table. Please allow extra space.

    2. Creswell (2007) recommends using at least two methods of the seven you recommended to increase the trustworthiness of qualitative data. You might want to consider this as a benchmark.

    3. You did not address the “numbers” issue. Since most participants in a study answer open-ended questions, saying the number of people who said something does not mean the same thing as in quantitative research. Wolcott’s word says that qualitative work should be reported in broader categories such as most respondents, some respondents, a few respondents, etc. This should be acceptable given the open-ended nature of qualitative inquiry.

    4. I think SSWR needs to make efforts to reduce what may be bias toward qualitative research. Some seem as if they assume that something reported in numbers is automatically better than something reported in words. As you know, there can be bias in either type of method. This needs to clearly communicated.

    5. The journal should send qualitative study manuscripts to reviewers who are experienced in using qualitative research. Can you work toward that? At the very least, how will you make sure that reviewers understand the document concepts? How will you make sure they can apply them well? For example, researchers should not get reviewers suggesting they run t-tests or chi-squares on qualitative data (Yes, that happens).

    6. How will you assure that these same standards are applied to reviews of qualitative work submitted for SSWR conference paper/workshop proposals? This is as important as the manuscript standards.

    I am pleased SSWR is working on this. Thank you for your efforts. I hope these comments are helpful.

    Joanne Riebschleger

  9. Rosalie Kane

    I am pleased to see this effort. It seems to me the most important overall message is the qualitative research has rigor–if it is observation, there is rigor in the sampling of observations, Sample selection has rigor as well.
    Section 1: Acknowledge that often qualitative work is done as part of quantitative including experimental research (to help explain the quantitative results) and the authors should be clear about that as part of the background.

    3.5.1 The section on coding seems to suggest that coding is not usual. It seems to me that coding is almost always done in some form. It also requires justification of more than one coder and showing their reliability. The latter is important but in fact it is also important to use multiple people in the development of codes. I think the key instruction here should be that the “Steps in developing and testing coding processes be described, as well as the inter-coder reliability at all steps.

    4.2 on quotations. I agree that manuscripts that are one quote after another are often problematic but I don’t think you should have suggested that authors go easy on quotations. Let them decide what is needed to illustrate the theme.

    Not sure where this goes, but there are several other types of qualitative research that have their own dynamics. Case studies at the program level, sometimes involving multiple state and agencies, can be large studies. When reporting on them, the authors should, I think, present the overall framework of the questions they were trying to answer through the cases.
    Focus group studies are also qualitative research and should be used when appropriate to explore an issue as opposed to reporting on a population or sample. No percentages of responses should be included in focus group reports.

  10. Jay Hedgpeth PhD, MSW

    Excellent
    I have only one concern. I would use issue or situation in place of problem. We may wish to evaluate positive situations.
    2.3 Note limitations in methods in addressing complexities or nuances of the problem

  11. Elizabeth Rohan

    Thank you for this effort and for giving others the opportunity to comment. I would like to add that these quidelines should also be distributed to reviewers. One of my frustrations with submitting qualitative work (to various journals) is that often reviewer comments indicate a complete lack of understanding of qualitaitve research. Interestingly, the comment is often about limiting the using quotes in the results where a summary would have sufficed. I agree wholeheartedly with others that authors not be discouraged to present our data (direct quotes from our participants) any more than authors of quantitative papers would be asked to suppress their data. How else will readers be able to determine if our summaries and inferences have any merit. Of course, as others have said, the results should not just be quote after quote. Expert weaving of themes and connecting of ideas throughout the results section is, of course, required.

  12. Evan Mayo-Wilson

    Great to see the journal taking steps to improve reporting quality! I’m sure the Editors are aware of the guidelines that exist in this area (http://www.equator-network.org/reporting-guidelines/coreq/; http://www.equator-network.org/reporting-guidelines/srqr/). If you haven’t already, I’d also suggest looking at what NICE, AHRQ, and other guideline developers use to evaluate quality – addressing the specific items in those instruments might improve the likelihood that a piece of research gets used to inform practice.

  13. Debbie Gioia

    These guidelines will make the manuscript reviewer’s task much easier and I applaud these efforts. The only thing that I did not see was a section for researcher reflexivity and/or researcher standpoint/lens. How this researcher(s) came to do this study, and how they are connected to the respondents is very important to me as a reviewer. Critical use of first person reflection and reporting of data is a point that I feel needs to be added to offer guidance to the author about how this manuscript might appear different from a quantitative manuscript. Should that be a separate section? or woven throughout? Also I was hoping to see something about novel reporting of data displays, and the journal’s appreciation and advice for photographic or document display within the body of the manuscript, if that is part of the qualitative method.

  14. Valandra

    I agree with several of the previous posts, particularly related to the journal’s efforts to provide qualitative researchers with guidelines that are based on qualitative research standards and criteria for establishing rigor, credibility, and trustworthiness. I too think this same effort should be made in getting reviewers for the journal and for reviewing qualitative research abstracts for the JSSWR conference.
    With respect to the manuscript guidelines for qualitative research, here are my thoughts and suggestions:
    3.4 I can understand wanting to know the training and background of the data collectors but given that with qualitative research, the researcher(s) is(are) the instrument, why request background and training info regarding data collection only. Instead, include a section in the manuscript format for “Researcher(s) Reflexivity” so that researchers can identify their standpoint, position, etc in relation to the whole study, including its design, data collection, analysis, etc. The researcher’s position is critical to the context and should be explicitly identified in both quantitative, which is just as subjective, and qualitative research.
    4.2 The use of the word ‘overuse’ of quotes is problematic. A hallmark of qualitative research findings are the quotes of study participants – instead, a measure of rigor and credibility would be to ensure that quotes adequately and accurately represent the theme, topic, concept being conveyed. Quotes are particularly pertinent in narrative, phenomenological ethnographic, feminist, critical, and action research were voice is more critical to understanding the phenomeon. I think the threat to rigor is the underuse not ‘overuse of quotes. Quoted material should include enough context to interpret.
    5.5 I appreciate the recognition that the lack of generalizability is not a limitation of qualitative research.
    Lastly, given that culturally responsive social work practice and policy is promoted as “best practice” in our profession, the same should apply to research – both quantitate and qualitative. I think manuscript guidelines for both qualitative and quantitative researcher should include specific standards to demonstrate culturally responsive methods as a matter of course the same way it is expected that researchers will provide detailed reports of their data analysis methods. This could be achieved by expanding the methods section of the manuscript to include specific measures particularly in studies involving cross-cultural research. Or at a minimum, researchers should be able to articulate in the limitations of a cross-cultural study, how their study does not meet standards of culturally responsive research.

    All of these suggestions would, of course, require an expansion of the number of pages for submission.

  15. Regina Praetorius

    Related to research credibility, Patton’s framework provides a useful guide for establishing credibility of a study that I suggest be included. It is delineated in:

    Patton, M. (1999). Enhancing the quality and credibility of qualitative analysis. Health Services Research, 34, 1189-1208.

  16. Joseph Richardson

    I think since more qualitative studies are being conducted by research teams and that qualitative research requires more cultural competent researchers, it is important to get an understanding of the background of the research team and how they were qualified to conduct this research. I would also like to see researchers identify the threshold for determining when an emerging finding is thematic is it 20 percent of the sample, 30 percent? Many researchers include themes that really are not, they may be novel findings but not themes. If two people in a sample of 25 have similar nuanced comments is that a theme? I also agree that there needs to be space for researchers to be reflexive and to discuss their positionality in the study, particularly for scholars of color, who conduct research in communities of color. The positionality of the scholar of color is rarely discussed in qualitative research.

  17. Julie Cederbaum

    I think these guidelines are well written but do want to point out that there are criteria for the presentation of qualitative work. These are criteria used in most medical/health journals. Please see the link below to the work of Tong and Colleagues – Consolidated criteria for reporting qualitative studies (COREQ): 32-item checklist
    http://intqhc.oxfordjournals.org/content/intqhc/19/6/349.full.pdf

  18. Marybeth Shinn

    These generally seem excellent and thorough guidelines. I agree with many other posts, particularly about use of quotations. Here is an issue less mentioned: Both quantitative and qualitative research are conducted within particular contexts and may not generalize beyond those contexts. All studies should strive to identify the contextual limits of findings, or at least to specify the features of their own context that might be important. Further, it is not numbers that lead to generalizability within a particular context, but representative sampling, so I appreciate the attention to sampling in 3.3. Sampling of material is important as well as sampling of people. Journalists may be interested in “man bites dog” stories, but social scientists (whether qualitative or quantitative) should endeavor to clarify when their findings are unusual or common within a particular context.

  19. Byron Powell

    I echo others’ applause for this effort to develop clear guidelines for qualitative research. I am aware of another set of guidelines that can be found below. Forgive me if others have mentioned these already.

    http://www.biomedcentral.com/authors/rats

    Apparently they were modified for Biomed Central and originally reported in the following publication:

    Clark JP: How to peer review a qualitative manuscript. In Peer Review in Health Sciences. Second edition. Edited by Godlee F, Jefferson T. London: BMJ Books; 2003:219-235

  20. Tatiana Masters

    I add my voice to the chorus appreciating this effort to lay out guidelines for reporting on qualitative research.

    I suggest elaborating this element in section 4, Results or Findings, further: “presentation should move beyond the level of description-only”. I often review (or even read published in journals) qualitative work that just lists themes, and it always disappoints. Please encourage authors to look at themes from a higher level, see which and what types of relationships exist among them all, and elucidate these patterns for the reader. I say to qualitative analysis students that they should not stop after gathering a pile of bricks, rather, they should build a structure with them.

    A minor addition to 3.5, Analysis, would be to state whether and how interviews were transcribed, and who did so.

  21. Jay Marlowe

    Thanks for developing these guidelines. I agree with many of the previous comments. Most particularly, I would not encourage authors to necessarily limit the data that they present — more often than not my critique when reviewing is that there is not enough evidence presented to justify the interpretation. I think it would be better to state that authors need to ensure that they present sufficient evidence to support their analysis/interpretation/deconstruction — this presentation will in some ways also be influenced by the chosen methodology.
    Section 1: is it possible to also include contributions to understanding theory? Not all qualitative research needs to be applied.

  22. Brandy Maynard

    I would also like to commend the journal for the efforts of proposing guidelines for qualitative research. In addition to the excellent suggestions provided thus far, my comments are as follows:
    3.2 I would like to second (or third) the need for authors to include information about researcher/research team reflexivity, characteristics and background of the researcher/team, and the nature and length of their relationship to the participants.
    3.3 I recommend adding reporting of sample size, sample characteristics, and participant dropout or refusals.
    3.4 I would also like to see information about the setting included- where was the data collected and over what time period?

  23. Roberta Sands

    I appreciate this undertaking, all the work that went into it, and the elicitation of feedback. I do worry, however, about being too prescriptive about what a qualitative article should look like and how it should be assessed.
    There is a bias in the guidelines toward interview-based research that is thematically analyzed. What about methodological articles? Ethnography?
    I would hope that the journal would be flexible about article length. Qualitative articles tend to be longer than quantitative ones. (The first thing I look at when considering sending an article to a specific journal guidelines is page numbers.) Considering that this is an online journal, space should not be a problem. On the other hand, if the article is too long, reviewers are burdened.
    I concur with a comment by one of the respondents about reflexivity. It is desirable for authors to describe where they are coming from and how their possible biases and positionality in relation to the researched affected the findings.
    1.1 I would describe the research problem.
    1.2 Besides linking the research questions to social or health problems, one can link them to populations served by social workers or to social services.
    2.1 How recent is recent? Is recent better than old, seminal articles or books? I think, too, that it is desirable to review both quantitative and qualitative literature.
    2.4 Along with summarizing strengths and limitations of previous studies, provide a rationale for the study you are presenting.
    3. I would call this Methods, not Method.
    3.1 IRB is important but should not be the first item within Methods.
    3.3 Add “If this is an interview study” prior to first sentence (“Provide a detailed…”). I would add this statement to this paragraph: Give a rationale for the sample size.
    3.4 First line: add “focus groups” within parentheses. Note that there are other methods of triangulation besides data triangulation that can be used (e.g., investigator, theoretical, etc.). If multiple methods of data gathering are used, the author should explain how they work together or complement each other.
    3.5 Not all analyses are thematic. What about narrative analysis?
    3.5.2 Instead of “data analytic software” I would write “qualitative software.” The researcher does the analysis–not the software.
    4.2 Overuse would include multiple examples of the same theme where the examples are repetitive and do not add anything. As people have said, of course, you need quotes. I would not just list them (as I have seen in several manuscripts).
    5.2 I would add to “advance the literature” “advance theory.”

  24. Yin Ling Irene Wong

    I thought the guidelines are very thorough and well written. But these should be guidelines only; there needs to be flexibility for articles to deviate from the guidelines according to the specific studies being undertaken. Thanks.

  25. Hee-Choon Shin

    Let us use “snowball/respondent-driven” instead of ‘snowball” in 3.3 Recruitment and Sampling. The original “snowball sampling” is as scientific as any other random sampling from a population (e.g., Goodman, Leo A. 1961. “Snowball Sampling.” Annals of Mathematical Statistics 32:148-170.). Indeed, a research based on the original “snowball sampling” should be classified as “quantitative”.

  26. Ioana Schmidt

    I echo the other commentators in applauding the effort that went into this guidelines and the utility they can offer qualitative researchers. Overall, extremely helpful in minimizing ambiguity in qualitative submissions while also allowing for a broad range of work. Two minor considerations:
    1. Perhaps consider language allowing for a chart or concept map prior to the Results, such as the Methods section, as these can also sometimes illustrate sensitizing concepts or areas of focus.
    2. The guidelines appear to be requesting a specific qualitative framework (ethnography, phenomenology, etc). I would suggest leaving this a bit more open, as not all qualitative work fits neatly into one or two of these paradigms.

  27. Lauren Gulbas

    Kudos to the editors for their efforts to develop guidelines in qualitative research. One of the hallmarks of qualitative research is the notion of saturation. Saturation is a guiding principle in both sampling and analysis, yet despite its importance, little attention is directed toward the way(s) in which saturation is achieved. Guidelines that prompt authors to move beyond simple statements about saturation (e.g., “saturation was achieved”) would help to facilitate reviewers’ and readers’ ability to ascertain rigor. Authors should be encouraged to engage in critical reflection about how saturation was carried out, documented, and achieved in ways that speak to the strengths and limitations of the study.

  28. Carol Cleaveland

    I appreciate the efforts of the committee to clarify guidelines and to ensure rigor in qualitative research. I agree with some colleagues here that quantitative research can not be assumed to be bias free and certainly shouldn’t be assumed to be, ipso facto, more rigorous. My hope is, as well, that qualitative work will be evaluated by those with experience in these methods. On another point regarding quotes, I must say that if these are limited we have lost the benefit of rich, thick data as well as the voices of our participants. How can our analysis be meaningful if we are limited in the presentation of the data that informs these ideas?
    I have one concern that has not been raised by anyone as of yet, which is in the presentation of ethnographic data. Like most ethnographers, I have been trained to write field notes immediately after data collection to mitigate distortion by memory and to ensure that the notes are detailed and rich. I have, at times, used quotations from these field notes as one might a quotation from a respondent. This practice, I argue, gives the reader a view of the scene as it enfolds that isn’t captured as well in a description summarizing what happened. I would like the committee to include guidelines on the presentation of ethnographic research.
    Thank you for your consideration.

  29. Jeffrey Draine

    My initial reaction was “sheesh! must we dualize our methods so?” I’ve taken to asking students to re-write concept papers without either of the “q” words to get them out of this habit of referring to methods with either-or language. There are plenty of examples where methods blend–and where a host of approaches and analytic strategies may be used. The point is to describe them in sufficient detail. In this way the same standards of rigor and transparency apply to all work, and I think this statement should say so. Having stopped pretentiously rolling my eyes enough to read the document, I agree with many comments about the diligent attention the authors of the document have paid to this task–and the comments of many of the respondents, scholars to whom I owe a lot of deference. My suggestion is simply to add some attention to the idea that this is not a dualistic–or mutually exclusive division in our scholarship–and any guidelines ought to clearly refer to overall standards for scholarship as their anchor.

  30. Ian Shaw

    I have made some detailed comments. Hope they are helpful. Draft in lower case. my comments in upper case

    3.3…For example, if using phenomenology, a sample of eight to 10 participants might be appropriate, whereas a larger sample size NORMALLY would be expected with grounded theory
    (SEE ALSO MY COMMENT ON THE FINAL CLAUSE 5.5). ALTHOUGH THESE ARE GIVEN AS EXAMPLES THEY ARE THE ‘EASY’ EXAMPLES. IF SOMEONE HAS DONE NARRATIVE RESEARCH THE QUESTION OF SAMPLE SIZE MAY BE MORE DIFFICULT. EVEN MORE SO, IF IT IS ANALYSIS OF DOCUMENTS OR ARCHIVES THE QUESTION BECOMES NOT ONE OF NUMBERS IN A SAMPLE BUT A QUESTION ABOUT THE NATURE (RANGE, EXTENT ETC) OF THE ‘CORPUS’ OR ‘ARCHIVE’ FROM WHICH IT IS DRAWN. THE RELEVANT IDEA IN SUCH CASES IS SAMPLING OF MEANINGS RATEHR THAN OF PARTICIPANTS. THIS IS PART OF THE WIDER QUESTION OF HOW CONSIDERATIONS OF SAMPLING HAVE A DIFFERENT AND LESS PERVASIVE STANDING IN QUALITATIVE RESEARCH THAN IN QUANT. SOME OF THE WORK ON CASE SELECTION IS RELEVANT HERE. I WOULD RECOMMEND THAT THE TERM ‘SAMPLE’ IS NOT USED BUT THAT A TERM SUCH AS ‘SELECTION STRATEGY’ IS USED. THIS REMOVES THE RISK THAT THE REQUIREMENT GETS UNHELPFULLY LINKED TO ITS ORIGINS AND THE STRING OF MATHEMATICAL NOTIONS FROM WHENCE IT CAME.

    3.4 Include information regarding who collected the data and their training or background.
    WHILE I DO NOT WANT TO RESIST THIS ENTIRELY I AM CONCERNED ABOUT THE ASSUMPTION THAT SEEMS TO BE THAT RESEARCHERS WILL ALWAYS BE BETTER IF THEY HAVE HAD SOME SORT OF TRAINING. THIS IS DIFFICULT FOR THREE REASONS. FIRST I HAVE NEVER HAD TRAINING IN QUALITATIVE METHODS. IF THAT IS THE CASE FOR ME THEN… SECOND, THERE ARE IMPORTANT QUESTIONS ABOUT HIERARCHIES OF EXPERTISE THAT I SUSPECT ARE IMPLIED HERE. QUALITATVE METHODOLOGISTS HELPFULLY HAVE CHALLENGED ASPECTS OF THIS (THOUGH SOCIAL WORK SCHOLARSHIP HAS POORLY ENGAGED WIITH THIS AREA). THIRD, A SIMILAR POINT ARISES IN CONNECTION WITH THE EMANCIPATORY RESEARCH TRADITION WHERE THE NOTION THAT (ACADEMIC) TRAINING IS NECESSARY WOULD BE DEEPLY MISTRUSTED.
    3.4 Highlight data triangulation
    I HAVE A CONCERN HERE. ‘TRIANGULATION’ IN THE QUANT TRADITION MEANS SOMETHING VERY DIFFERENT FROM THE QUAL TRADITIONS (PL). I SUSPECT YOUR REVIEWERS MAY INTERPRET THIS REQUIREMENT IN REALIST TERMS AND IN ACCORD WITH THE EARLY WORK ON TRIANGULATION BY CRONBACH ET AL. POST DENZIN THIS IS DIFFICULT. WHILE I KNOW THERE SEEMS TO BE LITTLE SYMPATHY FOR DENZIN’S POSITION/S IN SSWR, IT IS A POSITION OF INFLUENCE. HE SPEAKS OF TRIANGULATION, WHICH IS MULTI-METHOD IN FOCUS AND ‘REFLECTS AN ATTEMPT TO SECURE AN IN-DEPTH UNDERSTANDING OF THE PHENOMENON IN QUESTION. OBJECTIVE REALITY CAN NEVER BE CAPTURED. WE KNOW A THING ONLY THROUGH ITS REPRESENTATIONS. UNLIKE EARLY FORMULATIONS OF TRIANGULATION, TRIANGULATION IS NOT A TOOL OR STRATEGY OF VALIDATION, BUT AN ALTERNATIVE TO VALIDATION.’ IT IS ‘BEST UNDERSTOOD … AS A STRATEGY THAT ADDS RIGOR, BREADTH, COMPLEXITY, RICH¬NESS AND DEPTH TO ANY INQUIRY’ (DENZIN AND LINCOLN 2005: 5).
    Denzin, N. K. and Lincoln, Y. 2005. ‘The discipline and practice of qualitative research’ in N. K. Denzin and Y. Lincoln (Eds.) Handbook of Qualitative Research. Thousand Oaks: Sage Publications.
    3.4 note limitations related to using a single form of data collection at one time point

    I AGREE. BUT AGAIN I DO NOT AGREE WITH THE WIDELY HELD ASSUMPTION THAT RESEARCH IS ALWAYS BETTER IF THERE IS MORE THAN ONE METHOD. I WONDER IF AN RCT STUDY WOULD BE REQUIRED TO NOTE THE LIMTATIONS OF RELYING ON A SINGLE METHOD. I RECOMMEND SAYING ‘NOTE ANY POSSIBLE LIMITATIONS…’ – AND ADDING SAME REQUIREMENT TO ALL REVIEW GUIDANCE

    3.5 …reader to get a sense of how themes were established

    THOUGH OF COURSE THEMATIC ANALYSIS IS NOT THE ONLY FORM OF QUALITATIVE ANALYSIS. SEE ALSO 4.1 FOR SIMILAR POINT

    3.5.1 triangulation of data, and member checking
    THIS CONFIRMS MY SUSPICIONS THAT ‘TRIANGULATION’ IS BEING USED IN THIS DOCUMENT ONLY IN REALIST WAYS THAT ARE NOT ACCEPTED BY MANY QUALITATIVE RESEARCHERS. LIKEWISE FOR MEMBER CHECKING. THIS IS A DEVICE THAT HAS ATTRACTED SIGNIFICANT – IF TOO RARELY RECOGNIZED EVEN BY QUALITATIVE SOCIAL WORK SCHOLARS – CRITICISM

    4.1 unless the study is purely descriptive
    ‘PURELY’ IMPLIES A SOMEWHAT PEJORATIVE LIMITATION? THERE ARE THOSE WHO ASSUME RESEARCH THAT SEEKS TO EXPLAIN IS SUPERIOR TO THAT WHICH SEEKS TO DESCRIBE AND UNDERSTAND. THIS TO ME IS A DEEPLY UNHELPFUL UNDERSTANDING OF THE NATURE OF THE SCIENCE ENTERPRISE IN SOCIAL WORK.

    4.3 Including a graphic or schematic chart can be helpful
    I SUGGEST ‘…MAY BE HELPFUL’

    5.1 Synthesize the findings and link them to the research questions
    AGAIN THIS IS TOO SIMPLE. ‘FINDINGS’ – A TERM INCIDENTALLY I DO NOT LIKE THOUGH I OFTEN USE IT – MAY BE ALL THE MORE SIGNIFICANT WHEN THEY RESIST SYNTHESIS. THIS IS AN ESTABLISHED REALIZATION GOING BACK AT LEAST AS FAR AS WORK BY TREND IN THE 1970S
    (‘FINDING’ IMPLIES THAT SOMETHING IS THERE HIDDEN AND WAITING TO BE ‘FOUND’)

    5.3 Describe the ways in which the findings contribute to the knowledge bases for practice or policy
    BUT WHY SHOULD ALL SOCIAL WORK RESEARCH HAVE A DISCERNIBLE APPLICATION? LIKE SEVERAL REQUIREMENTS IN THIS DRAFT DOCUMENT CONDITIONALITY IS NEEDED

    5.3 Describe the ways in which the findings contribute to the knowledge bases for practice or policy
    5.4 Explain the implications of the findings for practice, research, and/or public policy
    WHAT IS THE DIFFERENCE BETWEEN THESE REQUIREMENTS? YES I KNOW IT SAYS DESCRIBE AND EXPLAIN BUT IN PRACTICE I FIND IT HARD TO KNOW HOW WOULD UNDERAKE TWO DIFFERENT TASKS IN THIS WAY.

    5.5 Discuss the strengths and limitations of the methods in the context of your research perspective. For example, limitations related to sampling should be identified within a qualitative framework. The sample size or lack of depth in the data might limit transferability; however, a lack of generalizability is not a limitation in qualitative research
    I SUSPECT THIS HAS BEEN ADDED ON AS A CAVEAT DURING THE DEVELOPMENT OF THESE GUIDELINES IN PARTIAL – BUT IN MY VIEW INCOMPLETE – RESPONSE TO CONCERNS I GIVE ABOVE. AT LEAST IT NEEDS EMBEDDING IN THE RELEVANT PART OF THE GUIDANCE.

    IN GENERAL IT WILL BE SEEN THAT MY ASSESSMENT HAS A COMMON THREAD OF QUESTIONING THE EXTENT TO WHICH THESE QUIDELINES ASSUME SHARED CRITERIA, STANDARDS ETC BETWEEN QUALITATIVE AND QUANTITATIVE RESEARCH

  31. Michael A. Lewis

    I know we’re supposed to be commenting on the guidelines but since Valandra proposed (if I correctly understood) that both qualitative and quantitative researchers be required to indicate how the methods/measures they used were culturally responsive, I figured it’s appropriate for me to respond. As someone who uses quantitative methods in my work, I’m not sure how I’d indicate that a statistical model or technique I used was culturally responsive. This is because I’m not clear on what “culturally responsive methods/measures” means within the context of quantitative research. I’m not clear on what it means within the context of qualitative research either but perhaps those more familiar with qualitative research are. So I think before taking Valandra’s advice regarding this issue, a similar process of creating guidelines on what it means for qualitative and quantitative research methods to be culturally responsive may need to take place.

  32. Michael A. Lewis

    It sounds like Joseph Richardson is saying that since qualitative researchers need to be more culturally competent than quantitative ones, qualitative researchers who submit their work to the journal should be required to indicate their qualifications, including specifically how culturally competent they are, to have conducted the research. First, I question the idea that qualitative researchers need to be more culturally competent than quantitative researchers. I’m not necessarily saying that JR is wrong about this. But it would be good to hear some elaboration. Second, I’m a bit uncomfortable with the idea of the journal getting to a point where acceptance for publication in part depends on how culturally competent the researcher is. It seems to me that whether or not a paper is accepted for publication should depend on the quality of the work. I can see how a critique of a qualitative study’s methods, depending on what the study is about, might involve the assessment that the researcher failed to take into account a particular cultural perspective. But I’m not convinced that the journal should try to assess whether the researcher is qualified, that is culturally competent enough (whatever this means), to have conducted the research in the first place.

  33. Will Hall

    I believe this will be very helpful. I just have a few comments:
    1.3. Are quantitative researchers asked to discuss the compatibility of their research questions with quantitative inquiry? I think this might be a double-standard.
    3.3. It might also be good to ask authors to explain why their sample size is sufficient given their research questions. Small sample sizes are acceptable in certain situations, such as interviewing participants in a narrow target group like clients in an intervention trial, doing a longitudinal data with many time points, conducting an exploratory study with a new and unique phenomenon, or interviewing a specific and/or hard to reach population group that is not represented in the literature.
    4. Not all qualitative results involve a presentation of “themes” per say, such as theory development using grounded theory.

  34. Paula Allen-Meares

    Thank you for allowing me to comment. Great job.
    3.3 – Shouldn’t the researcher provide a rationale for providing the criteria for who/what is in the study?
    3.5.1 – There is no mention of the training of the coders to achieve high reliability.
    3.5 – There is no mention of the triangulation of data – the importance and potential.
    3.5.1 – Theory can be useful in explaining results. This point needs to be emphasized.

  35. Shaun Eack

    Excellent initiative. I would recommend expanding the latter part of Section 3.5.1 on steps taken to increase the credibility of data collected, and moving it to its own section (3.5.3). These contents are essential guidance for authors that may be easily overlooked in the current draft, and do not occur exclusively in the context of data coding, as the current draft my imply.

  36. Valandra

    I appreciate Michael Lewis’ comments and questions about how to assess the cultural responsiveness of quantitative and qualitative methods. Several scholars have written about the importance of quantitative researchers ensuring that the scales, instruments, inventories they use are applicable with multicultural populations (See Royse, D. (2011) ). Its not uncommon for older assessment instruments to be tested and based largely on Euro-American populations and then administered to multicultural communities in which they may not be applicable. Responses to items of a scale measuring concepts such as “safe neighborhood”, “attitudes toward police officers,” or “moods” for example depending on the worldview, lived experiences of the population. So a measure of cultural responsiveness for a quantitative study might be ensuring instruments measure the same construct dependably for ALL groups – as a measure of cross-cultural or multicultural reliability. Assessing whether test scores from a specific measurement will relate to other already established cross-cultural measures or have the same degree of predictive validity for all groups is another way of assessing validity within the context of culturally responsive quantitative research. These are only a few examples. There are also ways quantitative researchers can demonstrate culturally responsiveness in ensuring that recruitment methods chosen are compatible or aligned with the culture and/or lived experiences of the population. Even the nature of the research question or hypothesis, whether quantitative or qualitative can demonstrate cultural bias depending on the population being studied. For example, assumptions about inherent dysfunctions of a population can feed into the type of hypotheses formulated or research questions developed and tested. Within the realm of culturally responsive qualitative research, social work can learn a lot from the field of education and psychology. The work of Dr. Linda C. Tillman (2006, 2002) comes to mind with respect to African American populations. She has published extensively in the area of education and the use of culturally responsive qualitative research methods and approaches with African Americans. In the field of social work, Dr. Priscilla Gibson & Dr. Laura Abrams (2003) have published on racial differences in engaging, recruiting, and interviewing African American women in qualitative research. I use these references frequently in designing and implementing my studies because my research focuses the experiences of African American families. Measures of cultural responsiveness are pertinent to engaging in ethical, responsible research (quantitative and qualitative ) that should minimally, leave communities and families better off or at least no worse off after the researcher is gone. Now some may argue that thats what we have IRBs for and I can appreciate and respect the role of the IRB in assessing ethical standards and “do no harm” in critiquing research studies and perhaps it may be appropriate to consider the expansion of IRB protocols to include more culturally responsive standards. Regardless, social work related journals could play a major role in changing the climate and environment for the ways social scientists practice research.

  37. Joan Blakey

    It is useful to have a guideline as long as there is room to adjust as needed. Qualitative manuscripts are different and sometimes require some flexibility. Most of the order that has been provided makes sense. I would put some guidelines in different places. For example: 1.3. I typically provide this rationale under methodology. I describe why I chose a particular method. I may also provide a rationale in the gaps in the literature section. I typically do not provide a rationale for the method in the introduction. 3.1 IRB statement is not the first thing I talk about in the methods section. I typically include it as part of the data collection procedures. 3.5.1 is actually rigor or trustworthiness and should be its own section (3.6. Rigor). 3.5.1. is typically discussed when talking about analysis. Section four is the Findings section. Most qualitative researchers don’t use results. We say findings. As for providing direction on the number of quotes, etc. I do not think this should be a part of the guidelines. I have had reviewers tell me too many, and that they want more for the same article. Provide the reader with general guidelines such as : The quotes do not speak for themselves and need to be set-up. The findings should include themes /overall findings and use the quotes to substantiate the findings. I love the rest of the guidelines. Hopefully, qualitative researchers will find these guidelines helpful.

  38. Joanne Yaffe

    Thanks for this opportunity to weigh in on the guidelines. I was a bit shocked to see that I was already credited, and so thank you for that, but now I feel like I really need to earn it. I have one typo to note, one comment on your search procedures, and a few comments on the guidelines themselves. All in all, I think this is a pretty good job and I will be eager to learn what other comments have been provided.
    1. Typo: Page 5, line 3 below table: “The BioMed Central guidelines are based Jocelyn Clark’s (2003) guidelines,” I think you probably mean *based ON*, right?
    2. Search: I was a bit puzzled that you went to sociology journals but not anthropology journals. Most of what I know about ehnolography and participant observation I learned through my minor in anthropology. However, as you wound up going with Google Scholar, I don’t think this is an issue.
    3. Under “IRB Statement.” I may be overly cautious here, but I have been surprised that respondents will read a publication that resulted from their participation and then file a complaint when they recognize themselves. So when the guidelines say: “If pseudonyms are used to protect the anonymity of participants, be sure to note this protection,” I think it is important to mention that it is important to ensure that participants cannot be readily identified from their description, from the events they relate, or from the language that they use.
    4. Under “Methods”: The guidelines say “Identify the methods used in the research (e.g., grounded theory, ethnography, or narrative research).” If these are not the only three, then you might want to leave it open by using e.g. or etc. Photovoice is already giving rise to similar methodologies. Amanda Barusch is proposing using individuals’ narrative writing in ways that I don’t yet fully understand, but don’t fall into those categories, even the “narrative research.” Just don’t want to close it off, right?
    5. Under “Recruitment and Sampling”: Why not say that sample size should be justified?
    6. Data collection is great and I especially like the Analysis guideline
    7. Under “Results or Findings”: I agree that a schematic or other illustration can be really helpful, but sometimes graphics are fully redundant with the text — nothing new is really learned — and so the graphic might not be necessary. Worse than that, though, some illustrations are more than difficult to understand. So it might be necessary to say that graphics should illustrate concepts and relationships in a way that the text cannot, and sometimes it may be necessary to walk the reader through the graphic so that it is easier to understand.

    I hope this wasn’t overkill, and that my comments are helpful in some way!

  39. Yvonne Smith

    Thank you for the opportunity to respond to these guidelines. I can appreciate the difficulty of developing protocol that fit the wide range of qualitative methods in our field. I agree with many of the points raised by respondents so far, but I did want to specifically respond to two points here. First, in response to section 3.1, I think the instruction to avoid detailed description of study sites is quite problematic for organizational ethnography. When processes of organizations themselves are the focus of a study–as is often the case with my own work–it is essential to provide a relatively detailed description of programs, client populations, policies, practices, and even at times the physical appearance of organizational spaces. Depending on one’s IRB provisions, it may even be appropriate to identify the organization by name. Or, one might, for example, inform individual research participants that, even though their personal identifying information will be disguised using pseudonyms, etc., and the organization will be also be given a pseudonym, the identity of the organization may still be discernible to some readers. What seems to me to be most essential here is that researchers follow the anonymity and confidentiality procedures spelled out in their consent documents, with the understanding that the specific procedures may legitimately vary depending on the study.

    And, second, I want to respond to section 1.3: “Briefly discuss the compatibility of your research questions with qualitative inquiry.” I struggle to imagine useful responses to this prompt that would address the use of “qualitative inquiry” in general. It would make more sense, I think, to direct authors to explain the fit of their _specific_ methodological approach in light of their particular research questions. And this work seems most suited to the methods section.

    I hope my comments are helpful. I really appreciate everyone’s effort to consider how best to report qualitative work.

  40. Cheryl Hyde

    How will mixed method studies be reviewed? many of us use both quantitative and qualitative methodologies, yet these guidelines present methods as if in discrete silos. I don’t know if its feasible for an author to cover all that is demanded by quantitative and the qualitative guidelines. Is it possible to create a more general or mixed method guide?

  41. Michael A. Lewis

    Valandra. Thanks so much for the clarification. I have a much better sense of what you were referring to and agree with everything you said. I think where we may differ (and I say differ instead of disagree on purpose) is how we refer to the important topics you raised. As just one example, suppose a researcher did a quantitative study using a measure which was tested for reliability and validity on a sample of all white men or another one did a qualitative study in which only 40 white men were interviewed. Suppose further that the quantitative researcher later used this measure on a sample of black girls without presenting any argument whatsoever for why such a practice is justified. Suppose the qualitative researcher interpreted their findings on their sample of 40 white men to apply to all people and suppose the study was about, say, participants’ perceptions of the police. You might refer to these as culturally unresponsive research practices. I wouldn’t disagree with you but my initial reaction might be to simply regard them as examples of very poor research methodology. I only bring up this semantic point because I wonder if the concerns we share require the journal to come up with new standards of culturally responsive research or can current standards, such as thinking carefully about whom you sampled and how this affects the inferences and interpretations you can make, suffice?

  42. Deborah Padgett

    I really appreciate reading all of the useful commentary here and wanted to add my two cents. I am working on the 3rd edition of my social work qualitative methods textbook and value having this and other input to keep it up to date. Since we qualitative researchers do not (have to) agree on everything, there are bound to be different opinions and guidelines are that much harder to produce (even when assured that they are guidelines, not rules).
    I agree with several points including: no need to caution about limiting quotes, remove Section 1.3 on rationalizing compatibility of qualitative inquiry with the research questions, assure parity in what JSSWR asks of qualitative vs. quantitative researchers, move the IRB mention to later under Methods, and be open to data displays other than lists of themes with illustrative quotes. I think we are past the time when we have to rationalize using qualitative methods but it doesn’t hurt to justify (at least in one’s mind) which qualitative method was used and why.
    Researcher/team reflexivity is important but where it goes and how it is described is not standard (ethnographers usually relegate this to a lengthy Appendix essay). Are there some good models out there that fit journal restrictions?
    Two concepts continue to be a challenge: triangulation and saturation. I agree that triangulation is over-emphasized as a realist standard yet believe that different sources of data (where appropriate) can bring more than a single source. This seems to be where Denzin has turned—viewing triangulation not as ‘gold standard check’ but as multiple perspectives of which no single one is hegemonic. Triangulation too often assumes corroboration and there trouble lies. Perhaps pull back from heavy use of the term in the guidelines? As to saturation, this seems to be a red flag for critics of qual. and almost always has to be set against the need for pre-planned sample sizes (to which can be added a caveat about some variation needed). It is invoked more for analysis purposes and here seems right. (Yes, I am a pragmatist.)
    The stickier issues come with trying to cover all approaches without become overly directive and encyclopedic. Agree that there should be more description of the setting when that is in play (e.g., ethnography or case study). Narrative, ethnographic and phenomenological approaches have become so diversified it has become a real challenge to cover all possibilities—compare the micro-structure and formalism of a conversation analysis with the sweeping arc of an ethnography! I vary my reports based upon the methods and intended audience and try to select the right journal and follow that journal’s guidelines. The good news for me is we are talking about this!

  43. Valandra

    Michael, I would describe the scenarios you presented as both culturally unresponsive AND a very poor research methodology. I don’t believe the two descriptions are mutually exclusive. Plenty of researchers think critically about the populations they sample and how their sampling and data collections affect the inferences and interpretations they make and it is sound to do so but doing so doesn’t necessarily preclude them from engaging in culturally insensitive research, some of which has dire consequences and repercussions for the populations and communities whom they engage in the research process.

  44. Fang-pei Chen

    Thank you for the invitation to comment on the guideline. It is difficult to follow model guidelines of quantitative research to come up with one for qualitative research because quantitative approaches are derived from one tradition. Yet, qualitative research is an umbrella term that covers many traditions. So I suggest construct a guideline that states the objectives to achieve in a qualitative research manuscript and those in each section of the manuscript, rather than an inventory of required writing components. I would suggest, preferably in the opening paragraph, acknowledge the various traditions of qualitative research, and stress that it is the authors’ responsibility to define and follow the rigor of their chosen tradition, and show clearly how it was done to the readers. Along the same line, I would suggest state the objectives of each section. For example, for Introduction, the objectives may include that the authors provide sufficient detail of relevant elements in the context of the research, justify the significance of the research question, and articulate the implications of this research. The guideline may give concrete examples as illustration, not requirement, of writing components such as describing study settings in case studies and ethnographic studies. However, what components to include in writing should be at authors’ discretion according to their methodology. The purpose of review then is to assess to what degree the authors achieve the objectives in their respective tradition.

    Knowing that perhaps the work group had already reviewed, I wish to share some references that I have found very helpful when I work on my own writing.
    Matthews, S. H. (2005). Crafting qualitative research articles on marriages and families. Journal of Marriage and Family, 67(4), 799-808.
    Morrow, S. L. (2005). Quality and trustworthiness I qualitative research in counseling psychology. Journal of Counseling psychology, 52(2), 250-260.
    LaRossa, R. (2012). Writing and reviewing manuscripts in the multidimensional world of qualitative research. Journal of Marriage and Family, 74(4), 643-659.

  45. Michael A. Lewis

    Valandra
    I agree that in the examples I gave as well as in general, research practices can be both culturally unresponsive and bad methodology. The question I was trying to raise is whether any culturally unresponsive research practice would also be bad research methodology. In other words, is cultural unresponsiveness in research simply poor methodological decision making? Of course, such poor decision making could be caused by racial bias, gender bias, etc. But the effect of such bias or insensitivity would be bad and culturally unresponsive methodological decisions. I don’t know the answer to this question. I’m just raising it. From your earlier comments, it sounds like you’re much more familiar with the literature on cultural responsiveness and social research than I am. Thus, maybe you can come up with a host of examples of culturally unresponsive research practices that aren’t also examples of bad methodology. I’m raising this question because it seems to me that the answer to it is relevant to whether the journal ought to adopt new standards of culturally responsive research or whether, as I suggested earlier, current standards regarding what constitutes sound qualitative and quantitative research methodology might suffice.

    In your initial posted comment, you referred to the cultural responsiveness (or not) of one’s choice of research question or topic. I do think this is a place where sound research methodology and cultural responsiveness or sensitivity can diverge. This is because choice of research question is less about research methodology and more about the values one brings to their research. I think your proposal for standards of culturally responsive research definitely makes sense here but I’d still want the journal to be very careful. I suspect there are some research questions that pretty much everyone, at least everyone involved with the journal, would agree are culturally unresponsive or insensitive. Is the “black-white educational achievement gap” a result of blacks being less intelligent than whites, is this gap because blacks have an inferior culture compared to that of whites, do women make less than men, on average, because the inferior female culture results in their striving less to attain high standards than men do? These are research questions I suspect all connected to the journal would regard as examples of culturally unresponsive research questions. But I imagine there might be questions where opinions about their cultural responsiveness might diverge quite a bit. It seems to me that any standard of cultural responsiveness regarding choice of research question would need to be able to weed out the obvious cases of cultural unresponsive questions but still allow researchers to explore a variety of questions even politically and socially controversial ones. This might not be such an easy standard to come up with. But, again, because you seem more familiar with this area than I am, you might be familiar with how other journals already do this.

  46. Bobbie Iversen

    So many thoughtful responses – makes it much easier to add and briefly comment! Kudos to JSSWR for this opportunity! Just a few thoughts from my perch (that of a researcher does ethnographic as well as other qualitative and mixed methods research)…
    1.2 Why are “social” problems differentiated from “health” problems? To my mind, health is also a “social problem” — or perhaps better yet, “social situation” or “social concern”.
    3.4 I thought it really important to describe the disciplinary backgrounds of the researchers who worked on a big ethnography I did, as they were varied and added to “triangulation.” I always dislike the term “training”, it seems too top down rather than collaborative, so I wouldn’t include that term here. IAN – I have never had any “training” in qualitative methods either — but I’d say I was “self-educated” through reading about the epistemology as well as methodology for various types of inquiry, for sure! I would also add at the end of this section the importance of aiming for maximum triangulation (which really means consideration of all points of view and positionalities)…not only of data sources but also of researchers, respondents, and analysts, whenever possible.
    3.5.1 I also intensely dislike the term “member checking” — there’s something too “researcher as expert” about that term as well. I far prefer Michael Burawoy’s (2003) notion of “valedictory revisiting.”
    Otherwise: I wholeheartedly endorse the comments about theory – social and behavioral theory (though arguably, these are likely false dichotomies as well) as well as social work-related theory. As earlier posts described, there’s need to add notions of “understanding theory,” “advancing theory,” etc.
    I also agree with having those who use qualitative methods more frequently do the reviews (which I think happens in my case)….and for continuing to reduce dichotomous thinking about research. The more we focus critically on what we’re doing and why, and then describe it as thoroughly as we can, the more likely our research will result in meaningful contribution to knowledge-building, theory-building, practice, programs and policies.

  47. Peggy Polinsky

    The guidelines suggested seem appropriate and comprehensive – if only the manuscripts I review could follow such guidelines! Even minimally. Many manuscripts are incomplete with insufficient information to know what the authors really did and really found out. One thing i would suggest is changing the order in the Discussion Section (Section 5). Having the limitations last is usually disheartening. Maybe move 5.5 to follow 5.2, and end with what is now 5.4. Thanks for your great work on this.

  48. Valandra

    Michael, if current standards regarding what constitutes sound qualitative and quantitative research methodology sufficed, from my perspective, there would be no need for the increasing numbers of articles, book chapters, etc. from research scholars recognizing the value and importance of culturally responsive research practice methods. For example, Rubin & Babbie (2013) devote a whole chapter to culturally competent research and include content not only on cultural bias in the formulation of research questions, but recruiting, retaining participants, confidentiality, sampling techniques, data analysis, measurements, and reporting. Engel & Schutt (2014) devote several pages of their text to what they refer to as diversity issues which include topics on relevance validity, research design, single-subject design, survey research, systematic measurement errors, instruments and measurements, as well as issues such as interviewer-respondent characteristics, etc. Tripoli & Potocki-Tripodi (2007) have, in their book on International Social Work Research: Issues and Prospects, included cultural competency topics related to instrument construction, adapting existing instruments, cross-cultural instruments, etc. Creswell (2014) discusses cultural responsiveness in relation to the use of theory in qualitative, quantitative, and mixed methods approaches to research.
    Given the extensiveness of the scholarship on the use of culturally responsive research methods, it would seem fitting for journals to also include or expand existing standards to assess the cultural responsiveness of a study submitted for publication. So, in short, current standards for assessing sound methodology do not suffice, in my humble opinion. Lastly, I don’t know what other journals do in providing standards or guidelines for cultural responsiveness but my educated guess is that they do nothing currently; and perhaps, if their thinking is that current standards for assessing soundness suffice, they likely never will do anything.

  49. Michael A. Lewis

    Valandra,

    Just to be clear, I’m not saying that current standards regarding what constitutes sound qualitative or quantitative research methodology suffices. I’m just raising the question of whether this is the case. I specialize in the economics of public policy and quantitative method. In order to stay current in these areas, most of what I read are in the areas of mathematics/statistics. You seem far far more familiar with the literature on culture and research methods than I am. I say all this to make it clear that I just don’t feel informed enough to come to a conclusion regarding the question I’m raising. But it seems to me to be an important question to raise as well as arrive at a satisfactory answer to before the journal decides to change it standards. I did go online to skim the table of contents from the recent edition of the Babbie (spelling?) and Rubin book you mentioned. After having done so, you appear to be quite right that they’re concerned about culturally responsive research practice going well beyond selection of research questions. It’s hard to tell how much of a change in standards might be warranted without having read the chapter (which I plan to do) as well as some of the other things you mentioned in your previous posts. But having skimmed the table of contents and considered some of the other things you’ve been saying, I’m open to the idea that some change in standards along the lines of what you’ve been suggesting might be warranted.

  50. Brian L. Kelly

    Wonderful process and dialogue going on here. Thanks to all who have participated. As technology advances and audio and visual data collection methods and forms of representation become more readily available, it seems important to consider their within these guidelines. When authors wish to include audio and visual data in their findings, will the journal be open to those desires and work with authors to ensure they are able to include such content within their articles and/or through hyperlinks? Additionally, how might the journal determine the quality of such data and whether or not it should be included? Curry & Abrams (2015) recently co-authored a compelling piece utilizing photo elicitation interviewing for the journal. Perhaps that particular review process holds some insight into how the journal might consider increased inclusivity and standards of research involving audio and visual data.

  51. Larry Palinkas

    The guidelines looks quite good and are consistent with guidelines for publication of qualitative studies used by other journals. The terminology used by qualitative and quantitative methodologists to described the same phenomenon (e.g., analysis versus deconstruction) often leads to confusion and remains a persistent issue. However, reviewers should be expected to understand the points of commonality and how they contribute to the overall richness of the publications in this journal.
    My only substnative comment concerns the IRB section (3.1). The fact that a protcol has undergone IRB review should suggest that details on procedures for obtaining informed consent and maintaining anonymity and confidentiality and safety are implied in IRB approval. Given that qualitative manuscripts are already challenged with expectations concerning limited page length, going into detail on these issues in an article probably is not that necessary.

  52. Laura Abrams

    I agree overall with points made 1.3- I do not see a need to justify the use of qualitative research methods. What we should look for is a good fit between the research questions and the approach in all studies, not just qualitative ones.
    Under 3.5.1 also think we need not assume there are co-coders, or that co-coding is essential to a rigorous study. Many people still analyze their own data, and do so brilliantly. I think more important, is that the authors are very transparent about how conclusions about the data were reached. Often this information is missing from articles that I review, and I almost always ask to know more.
    Under 3.5.2 — I am starting to wonder if it is at important to mention the type of software used in qualitative research. Coding software does little more than retrieve coded data- and most have the exact same functions. I personally don’t care when someone says they used software, it does not change my appraisal of the methods section in any way. Usually I see that authors overemphasize that point as if to falsely legitimize their research like “SPSS” or “STATA”. While potentially unpopular, I would suggest we remove the mention of software altogether from these guidelines. I don’t see it as important.
    I have, for many years, struggled to educate reviewers and colleagues in social work about how to assess qualitative research articles on their own merits: not compared to any other “standard”. I see these guidelines as helping to inform reviewers as much as authors. I want to thank JSSWR for attempting to produce what I see as a very flexible set of guidelines and opening this dialogue up to the field.

  53. Varsha Pandya

    I laud the effort that SSWR has put in to develop “Proposed Author Guidelines for Qualitative Research Manuscripts Submitted to JSSWR”. I believe this is a necessary endeavor and I want to congratulate SSWR to take the initiative. Having said that, I concur with those comments that qualitative inquiry is wide, varied, and need not be held to CONSORT and PRISMA standards. Qualitative inquiry has more open boundaries between its strategies (grounded theory, ethnography, and such) then quantitative designs (experiments, surveys and such). Components of more than one strategy can easily be found in the design of qualitative inquiry. Refer to the reports of qualitative research I completed published in 2002 (along with Dr. W. Gingerich) in ‘Health and Social Work’ that used ethnographic method and the one published in 2009 in ‘Journal of Evidence-based Social Work’ that used more eclectic qualitative research methodology. Both articles report on complying with Ethics of Research involving human subjects, how rigor of the study was enhanced, and include longer raw data in the reports. It is also important that codes, themes, or advanced schema developed to report findings are supported through adequate quotes from more than one data file where applicable.
    When studying a topic from the perspective of the studied, literature review prior to completing the study is not recommended. Knowing what others have found can bias the researcher to interpret the text of the studied from existing knowledge rather than discovering newer nuances to the topic of study. Several qualitative researchers recommend reviewing literature after the findings to examine the similarities with existing and newly emergent knowledge. Same is true for using a theoretical lens for the concepts used in the study. Inductive logic requires that theory be developed as a result of knowledge emerging from a qualitative inquiry. When a topic is studied widely, a literature review may be appropriate as qualitative methodology is developed, but it certainly cannot be a requirement in reporting qualitative inquiry.
    3.5.1 I believe that coding is an essential step in analysis of qualitative data.
    3.5.2 I believe is not necessary. Some qualitative researchers, I for example, do not use data analysis software, and do all analytic work in MS Word.

Leave a Reply