Formulating a good research question: Pearls and pitfalls

Address for correspondence: Dr. Wilson Fandino, Anaesthesia Department, St Thomas' Hospital NHS Foundation Trust, Westminster Bridge Road, Lambeth, London SE1 7EH, United Kingdom. E-mail: moc.liamtoh@onidnaf.nosliw

This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.

Abstract

The process of formulating a good research question can be challenging and frustrating. While a comprehensive literature review is compulsory, the researcher usually encounters methodological difficulties in the conduct of the study, particularly if the primary study question has not been adequately selected in accordance with the clinical dilemma that needs to be addressed. Therefore, optimising time and resources before embarking in the design of a clinical protocol can make an impact on the final results of the research project. Researchers have developed effective ways to convey the message of how to build a good research question that can be easily recalled under the acronyms of PICOT (population, intervention, comparator, outcome, and time frame) and FINER (feasible, interesting, novel, ethical, and relevant). In line with these concepts, this article highlights the main issues faced by clinicians, when developing a research question.

INTRODUCTION

What is your research question? This is very often one of the first queries made by statisticians, when researchers come up with an interesting idea. In fact, the findings of a study may only acquire relevance if they provide an accurate and unbiased answer to a specific question,[1,2] and it has been suggested that up to one-third of the time spent in the whole process—from the conception of an idea to the publication of the manuscript—could be invested in finding the right primary study question.[3] Furthermore, selecting a good research question can be a time-consuming and challenging task: in one retrospective study, Mayo et al. reported that 3 out of 10 articles published would have needed a major rewording of the question.[1] This paper explores some recommendations to consider before starting any research project, and outlines the main difficulties faced by young and experienced clinicians, when it comes time to turn an exciting idea into a valuable and feasible research question.

OPTIMISATION OF TIME AND RESOURCES

Focusing on the primary research question

The process of developing a new idea usually stems from a dilemma inherent to the clinical practice.[2,3,4] However, once the problem has been identified, it is tempting to formulate multiple research questions. Conducting a clinical trial with more than one primary study question would not be feasible. First, because each question may require a different research design, and second, because the necessary statistical power of the study would demand unaffordable sample sizes. It is the duty of editors and reviewers to make sure that authors clearly identify the primary research question, and as a consequence, studies approaching more than one primary research question may not be suitable for publication.

Working in the right environment

Teamwork is essential to find the appropriate research question. Working in the right environment will enable the investigator to interact with colleagues with different backgrounds, and create opportunities to exchange experiences in a collaborative way between clinicians and researchers. Likewise, it is of paramount importance to get involved colleagues with expertise in the field (lead clinicians, education supervisors, research mentors, department chairs, epidemiologists, biostatisticians, and ethical consultants, among others), and ask for their guidance.[5,6,7,8]

Evaluating the pertinence of the study

The researcher should wonder if, on the basis of the research question formulated, there is a need for a study to address the problem, as clinical research usually entails a large investment of resources and workforce involvement. Thus, if the answer to the posed clinical question seems to be evident before starting the study, investing in research to address the problem would become superfluous. For example, in a clinical trial, Herzog-Niescery et al. compared laryngeal masks with cuffed and uncuffed tracheal tubes, in the context of surgeons' exposure to sevoflurane, in infants undergoing adenoidectomy. However, it appears obvious that cuffed tracheal tubes are preferred to minimise surgeons' exposure to volatile gases, as authors concluded after recruiting 60 patients.[9]

Conducting a thorough literature review

Any research project requires the identification of at least one of three problems: the evidence is scarce, the existing literature yields conflicting results, or the results could be improved. Hence, a comprehensive review of the topic is imperative, as it allows the researcher to identify this gap in the literature, formulate a hypothesis and develop a research question.[2] To this end, it is crucial to be attentive to new ideas, keep the imagination roaming with reflective attitude, and remain sceptical to the new-gained information.[4,7]

Narrowing the research question

A broad research question may encompass an unaffordable extensive topic. For instance, do supraglottic devices provide similar conditions for the visualization of the glottis aperture in a German hospital? Such a general research question usually needs to be narrowed, not only by cutting away unnecessary components (a German hospital is irrelevant in this context), but also by defining a target population, a specific intervention, an alternative treatment or procedure to be compared with the intervention, a measurable primary outcome, and a time frame of the study. In contrast, an example of a good research question would be: among children younger than 1 year of age undergoing elective minor procedures, to what extent the insertion times are different, comparing the Supreme™ laryngeal mask airway (LMA) to Proseal™ LMA, when placed after reaching a BIS index

PICOT FRAMEWORK

Table 1 summarises the foremost questions likely to be addressed when working on PICOT frame.[1,6,8] These components are also applicable to observational studies, where the exposure takes place of the intervention.[1,11] Remarkably, if after browsing the title and the abstract of a paper, the reader is not able to clearly identify the PICOT parameters, and elucidate the question posed by the authors, there should be reasonable scepticism regarding the scientific rigor of the work.[12,13] All these elements are crucial in the design and methodology of a clinical trial, as they can affect the feasibility and reliability of results. Having formulated the primary study question in the context of the PICOT framework [ Table 1 ],[1,6,8] the researcher should be able to elucidate which design is most suitable for their work, determine what type of data needs to be collected, and write a structured introduction tailored to what they want to know, explicitly mentioning the primary study hypothesis, which should lead to formulate the main research question.[1,2,6,8]

Table 1

Key questions to be answered when working with the PICOT framework (population, intervention, comparator, outcome, and time frame) in a clinical research design

Population

Occasionally, the intended population of the study needs to be modified, in order to overcome any potential ethical issues, and/or for the sake of convenience and feasibility of the project. Yet, the researcher must be aware that the external validity of the results may be compromised. As an illustration, in a randomised clinical trial, authors compared the ease of tracheal tube insertion between C-MAC video laryngoscope and direct laryngoscopy, in patients presenting to the emergency department with an indication of rapid sequence intubation. However, owing to the existence of ethical concerns, a substantial amount of patients requiring emergency tracheal intubation, including patients with major maxillofacial trauma and ongoing cardiopulmonary resuscitation, had to be excluded from the trial.[14] In fact, the design of prospective studies to explore this subset of patients can be challenging, not only because of ethical considerations, but because of the low incidence of these cases. In another study, Metterlein et al. compared the glottis visualisation among five different supraglottic airway devices, using fibreroptic-guided tracheal intubation in an adult population. Despite that the study was aimed to explore the ease of intubation in patients with anticipated difficult airway (thus requiring fibreoptic tracheal intubation), authors decided to enrol patients undergoing elective laser treatment for genital condylomas, as a strategy to hasten the recruitment process and optimise resources.[15]

Intervention

Anaesthetic interventions can be classified into pharmacological (experimental treatment) and nonpharmacological. Among nonpharmacological interventions, the most common include anaesthetic techniques, monitoring instruments and airway devices. For example, it would be appropriate to examine the ease of insertion of Supreme™ LMA, when compared with ProSeal™ LMA. Notwithstanding, a common mistake is the tendency to be focused on the data aimed to be collected (the “stated” objective), rather than the question that needs to be answered (the “latent” objective).[1,4] In one clinical trial, authors stated: “we compared the Supreme™ and ProSeal™ LMAs in infants by measuring their performance characteristics, including insertion features, ventilation parameters, induced changes in haemodynamics, and rates of postoperative complications”.[10] Here, the research question has been centered on the measurements (insertion characteristics, haemodynamic variables, LMA insertion characteristics, ventilation parameters) rather than the clinical problem that needs to be addressed (is Supreme™ LMA easier to insert than ProSeal™ LMA?).

Comparator

Comparators in clinical research can also be pharmacological (e.g., gold standard or placebo) or nonpharmacological. Typically, not more than two comparator groups are included in a clinical trial. Multiple comparisons should be generally avoided, unless there is enough statistical power to address the end points of interest, and statistical analyses have been adjusted for multiple testing. For instance, in the aforementioned study of Metterlein et al.,[15] authors compared five supraglottic airway devices by recruiting only 10--12 participants per group. In spite of the authors' recommendation of using two supraglottic devices based on the results of the study, there was no mention of statistical adjustments for multiple comparisons, and given the small sample size, larger clinical trials will undoubtedly be needed to confirm or refute these findings.[15]

Outcomes

A clear formulation of the primary outcome results of vital importance in clinical research, as the primary statistical analyses, including the sample size calculation (and therefore, the estimation of the effect size and statistical power), will be derived from the main outcome of interest. While it is clear that using more than one primary outcome would not be appropriate, it would be equally inadequate to include multiple point measurements of the same variable as the primary outcome (e.g., visual analogue scale for pain at 1, 2, 6, and 12 h postoperatively).

Composite outcomes, in which multiple primary endpoints are combined, may make it difficult to draw any conclusions based on the study findings. For example, in a clinical trial, 200 children undergoing ophthalmic surgery were recruited to explore the incidence of respiratory adverse events, when comparing desflurane with sevoflurane, following the removal of flexible LMA during the emergence of the anaesthesia. The primary outcome was the number of respiratory events, including breath holding, coughing, secretions requiring suction, laryngospasm, bronchospasm, and mild desaturation.[16] Should authors had claimed a significant difference between these anaesthetic volatiles, it would have been important to elucidate whether those differences were due to serious adverse events, like laryngospasm or bronchospasm, or the results were explained by any of the other events (e.g., secretions requiring suction). While it is true that clinical trials evaluating the occurrence of adverse events like laryngospasm/bronchospasm,[16,17] or life-threating complications following a tracheal intubation (e.g., inadvertent oesophageal placement, dental damage or injury of the larynx/pharynx)[14] are almost invariably underpowered, because the incidence of such events is expected to be low, subjective outcomes like coughing or secretions requiring suction should be avoided, as they are highly dependent on the examiner's criteria.[16]

Secondary outcomes are useful to document potential side effects (e.g., gastric insufflation after placing a supraglottic device), and evaluate the adherence (say, airway leak pressure) and safety of the intervention (for instance, occurrence, or laryngospasm/bronchospasm).[17] Nevertheless, the problem of addressing multiple secondary outcomes without the adequate statistical power is habitual in medical literature. A good illustration of this issue can be found in a study evaluating the performance of two supraglottic devices in 50 anaesthetised infants and neonates, whereby authors could not draw any conclusions in regard to potential differences in the occurrence of complications, because the sample size calculated made the study underpowered to explore those differences.[17]

Time frame

Among PICOT components, the time frame is the most likely to be omitted or inappropriate.[1,12] There are two key aspects of the time component that need to be clearly specified in the research question: the time of measuring the outcome variables (e.g. visual analogue scale for pain at 1, 2, 6, and 12 h postoperatively), and the duration of each measurement (when indicated). The omission of these details in the study protocol might lead to substantial differences in the methodology used. For instance, if a study is designed to compare the insertion times of three different supraglottic devices, and researchers do not specify the exact moment of LMA insertion in the clinical trial protocol (i.e., at the anaesthetic induction after reaching a BIS index < 60), placing an LMA with insufficient depth of anaesthesia would have compromised the internal validity of the results, because inserting a supraglottic device in those patients would have resulted in failed attempts and longer insertion times.[10]

FINER CRITERIA

A well-elaborated research question may not necessarily be a good question. The proposed study also requires being achievable from both ethical and realistic perspectives, interesting and useful to the clinical practice, and capable to formulate new hypotheses, that may contribute to the generation of knowledge. Researchers have developed an effective way to convey the message of how to build a good research question, that is usually recalled under the acronym of FINER (feasible, interesting, novel, ethical and relevant).[5,6,7] Table 2 highlights the main characteristics of FINER criteria.[7]

Table 2

Main features of FINER criteria (Feasibility, interest, novelty, ethics, and relevance) to formulate a good research question. Adapted from Cummings et al.[7]

Novelty and relevance

Although it is clear that any research project should commence with an accurate literature interpretation, in many instances it represents the start and the end of the research: the reader will soon realise that the answer to several questions can be easily found in the published literature.[5] When the question overcomes the test of a thorough literature review, the project may become novel (there is a gap in the knowledge, and therefore, there is a need for new evidence on the topic) and relevant (the paper may contribute to change the clinical practice). In this context, it is important to distinguish the difference between statistical significance and clinical relevance: in the aforementioned study of Oba et al.,[10] despite the means of insertion times were reported as significant for the Supreme™ LMA, as compared with ProSeal™ LMA, the difference found in the insertion times (528 vs. 486 sec, respectively), although reported as significant, had little or no clinical relevance.[10] Conversely, a statistically significant difference of 12 sec might be of clinical relevance in neonates weighing

Feasibility and ethical aspects

Among FINER criteria, there are two potential barriers that may prevent the successful conduct of the project and publication of the manuscript: feasibility and ethical aspects. These obstacles are usually related to the target population, as discussed above. Feasibility refers not only to the budget but also to the complexity of the design, recruitment strategy, blinding, adequacy of the sample size, measurement of the outcome, time of follow-up of participants, and commitment of clinicians, among others.[3,7] Funding, as a component of feasibility, may also be implicated in the ethical principles of clinical research, because the choice of the primary study question may be markedly influenced by the specific criteria demanded in the interest of potential funders.

Discussing ethical issues with local committees is compulsory, as rules applied might vary among countries.[18] Potential risks and benefits need to be carefully weighed, based upon the four principles of respect for autonomy, beneficence, non-maleficence, and justice.[19] Although many of these issues may be related to the population target (e.g., conducting a clinical trial in patients with ongoing cardiopulmonary resuscitation would be inappropriate, as would be anaesthetising patients undergoing elective LASER treatment for condylomas, to examine the performance of supraglottic airway devices),[14,15] ethical conflicts may also arise from the intervention (particularly those involving the occurrence of side effects or complications, and their potential for reversibility), comparison (e.g., use of placebo or sham procedures),[19] outcome (surrogate outcomes should be considered in lieu of long term outcomes), or time frame (e.g., unnecessary longer exposition to an intervention). Thus, FINER criteria should not be conceived without a concomitant examination of the PICOT checklist, and consequently, PICOT framework and FINER criteria should not be seen as separated components, but rather complementary ingredients of a good research question.

Interest

Undoubtedly, no research project can be conducted if it is deemed unfeasible, and most institutional review boards would not be in a position to approve a work with major ethical problems. Nonetheless, whether or not the findings are interesting, is a subjective matter. Engaging the attention of readers also depends upon a number of factors, including the manner of presenting the problem, the background of the topic, the intended audience, and the reader's expectations. Furthermore, the interest is usually linked to the novelty and relevance of the topic, and it is worth nothing that editors and peer reviewers of high-impact medical journals are usually reluctant to accept any publication, if there is no novelty inherent to the research hypothesis, or there is a lack of relevance in the results.[11] Nevertheless, a considerable number of papers have been published without any novelty or relevance in the topic addressed. This is probably reflected in a recent survey, according to which only a third of respondents declared to have read thoroughly the most recent papers downloaded, and at least half of those manuscripts remained unread.[20] The same study reported that up to one-third of papers examined remained uncited after 5 years of publication, and only 20% of papers accounted for 80% of the citations.[20]

SUMMARY

Formulating a good research question can be fascinating, albeit challenging, even for experienced investigators. While it is clear that clinical experience in combination with the accurate interpretation of literature and teamwork are essential to develop new ideas, the formulation of a clinical problem usually requires the compliance with PICOT framework in conjunction with FINER criteria, in order to translate a clinical dilemma into a researchable question. Working in the right environment with the adequate support of experienced researchers, will certainly make a difference in the generation of knowledge. By doing this, a lot of time will be saved in the search of the primary study question, and undoubtedly, there will be more chances to become a successful researcher.

Financial support and sponsorship

Conflicts of interest

There are no conflicts of interest.