Publication of confirmatory studies required by Health Canada for drugs approved under a Notice of Compliance with conditions: a cohort study
=============================================================================================================================================

* Joel Lexchin

## Abstract

**Background:** Health Canada approves drugs based on limited data (Notice of Compliance with conditions [NOC/c]) and then requires companies to conduct confirmatory studies to validate the drugs' efficacy/effectiveness. The current investigation was carried out to determine whether these confirmatory studies are eventually published and are available to health care practitioners.

**Methods:** A list of drugs for which the confirmatory studies had been completed from 1998 to Sept. 30, 2014 was created from 2 published articles that listed NOCs/c and investigated whether they had been fulfilled, the NOC database and the NOC/c Web site. The confirmatory studies for these drugs were determined from Qualifying Notices, agreements between Health Canada and the drug companies. Possible publications from these studies were identified through a Web search, and companies were asked to confirm these publications. The time in days between fulfillment of the NOC/c and publication of the studies was calculated.

**Results:** There were 58 distinct confirmatory studies for 24 products made by 14 different companies. Eleven companies responded and identified 29 unique publications that reported on 31 studies. One company did not confirm a publication that was subsequently independently identified. Three companies did not respond, and in these cases another 18 publications were independently identified for an additional 19 studies. No publications were found for 7 studies. Thirty-one publications appeared a mean of 610 days before the NOC/c was fulfilled, and 17 appeared a mean of 572 days after fulfillment of the NOC/c.

**Interpretation:** Eighty-eight percent of the confirmatory studies were eventually published. Health Canada and drug manufacturers should take steps to ensure that knowledge about these publications is available to health care practitioners.

In an effort to ensure that promising therapies for serious illnesses can reach Canadians in a timely manner, Health Canada developed the Notice of Compliance with conditions (NOC/c) guidelines in 1998. The goal of this policy is to "provide patients suffering from serious, life threatening or severely debilitating diseases or conditions with earlier access to promising new drugs."1 An NOC/c could used for drugs with trials with only surrogate markers, drugs with phase II trials that require confirmation with phase III trials, or drugs with a single small to moderately sized phase III trial that requires confirmation of the efficacy or the safety of the agent in question.2 An NOC/c can be issued for a new drug or for a new indication for a drug already on the market.

A Qualifying Notice is the document that Health Canada sends to a drug sponsor indicating that the drug qualifies for an NOC/c. (Before February 2003, the Qualifying Notice did not exist, and a Letter of Understanding detailed the confirmatory studies. Henceforth, Qualifying Notice and Letter of Understanding will collectively be referred to as Qualifying Notice.) The Qualifying Notice outlines the additional clinical evidence to be provided in confirmatory studies, i.e., studies that definitively establish efficacy. When completed, these studies are submitted to Health Canada, and, if the studies are accepted, the product receives a full NOC. Should these postmarket trials not provide sufficient evidence of clinical benefit, the NOC/c could be revoked and the product removed from the market.3

Health Canada does not notify health care practitioners or the public that an NOC/c has been fulfilled except through the NOC/c Web site and, since September 2012, by including some information in the Summary Basis of Decision, a document that summarizes Health Canada's decision-making process in approving a new drug.4 Health Canada does not release any details about the completed confirmatory studies. Therefore, if health care practitioners, guideline developers or other interested parties want information about the studies, they have to rely on the studies' being published.

The current study examined drugs for which the NOC/c was fulfilled (i.e., confirmatory studies had been completed and accepted by Health Canada) to determine whether the confirmatory studies were published as full journal articles and the time between fulfillment of the NOC/c and publication of the articles. Secondarily, I looked at how long it took between the time the NOC/c was granted and when it was fulfilled.

## Methods

### Sources of data

A list of all drugs with an NOC/c whose conditions had been fulfilled and the date the conditions were fulfilled, from the time that the program started (1998) until Sept. 30, 2014, was compiled from 4 sources: articles by Lexchin3 and Law5 that listed NOCs/c and investigated whether they had been fulfilled, the NOC database ([http://webprod5.hc-sc.gc.ca/noc-ac/index-eng.jsp](http://webprod5.hc-sc.gc.ca/noc-ac/index-eng.jsp)) and the NOC/c Web site ([www.hc-sc.gc.ca/dhp-mps/prodpharma/notices-avis/conditions/index-eng.php](http://www.hc-sc.gc.ca/dhp-mps/prodpharma/notices-avis/conditions/index-eng.php)). The latter Web site is updated when an NOC/c has been fulfilled, although Health Canada does not specify what the timeline is between fulfillment and posting of the information on the Web site. In addition, the date on which the NOC/c was granted, the generic and brand names of the drugs, and the names of the companies marketing them were recorded.

Until the confirmatory studies are completed and the product receives a full NOC, the Qualifying Notices are publicly available on Health Canada's Web site. Once the conditions have been met, the Qualifying Notice is no longer publicly available, and therefore I requested these from Health Canada through the Access to Information Act. The list and description of the confirmatory studies were abstracted from the Qualifying Notice. The Qualifying Notices were independently screened by 2 people (J.L. and a family physician), and disagreements were resolved by consensus. Only confirmatory studies that provided information about drug efficacy/effectiveness were identified, as these are the ones that health care practitioners would be most concerned about. Studies looking solely at pharmacokinetics or pharmacodynamics were not examined. Additional information about safety that Health Canada required typically took the form of enhanced reporting of adverse drug reactions or was an additional requirement of efficacy/effectiveness studies. There were no required confirmatory studies that focused solely on safety.

A Web search was performed in the first week of October 2014 and was repeated in the first week of December 2015 and in the first week of February 2017 to determine whether a possible [clinicaltrials.gov](http://clinicaltrials.gov) identifier and/or journal publication(s) could be identified for each study listed in the Qualifying Notice. Matches between studies required in the Qualifying Notice and journal publications or trials registered in [clinicaltrials.gov](http://clinicaltrials.gov) were made on the basis of 1 or more of the following characteristics: generic name, number and particulars of trial participants (e.g., women with breast cancer), primary outcomes and description of the treatment. If no journal publication was given in [clinicaltrials.gov](http://clinicaltrials.gov), I searched PubMed and Embase for a journal publication. Terms used in the search depended on the level of detail in the Qualifying Notice about the required study. (See Appendix 1, available at [www.cmajopen.ca/content/5/2/E295/suppl/DC1](http://www.cmajopen.ca/content/5/2/E295/suppl/DC1), for an example of a search strategy.) Full journal articles published up to Feb. 6, 2017 were downloaded through the University of Toronto library Web site.

I then sent a letter to the drug manufacturers outlining the nature of the research, quoting a description of the confirmatory study or studies from the Qualifying Notices, giving the possible [clinicaltrials.gov](http://clinicaltrials.gov) identifier (if one was found) and the possible publication (also if one was found), and asking the company to confirm that the publication corresponded to the study or, if not, to provide a citation to a publication. After 1 month, a single reminder was sent to nonresponders. In cases in which companies did not confirm that a publication was matched with a study required under the Qualifying Notice or companies did not respond, publications identified using the strategy described above were used.

### Statistical analysis

I used descriptive statistics to report on the proportion of studies with corresponding publications. The time between when the conditions were met and journal publication was calculated in days, as was the time between granting and fulfillment of the NOC/c. The date of publication was as printed in the issue of the journal or the date published online. If only the month and year of publication were given, the publication date was deemed to be the midpoint between issues; for example, if a journal was published monthly, the day of publication would be the 15th of the month. Data are reported as means and were analyzed with the use of Prism 7.0 for Mac (GraphPad Software).

### Ethics approval

The York University Ethics Review Board waived ethics approval for this study, as only publicly available material was being requested from companies.

## Results

During the study period, there were 63 NOCs/c for 46 new drugs or new indications for existing drugs. Thirty-four NOCs/c were fulfilled for 26 products, 19 with an NOC/c for a single indication, 6 with 2 indications and 1 with 3 indications (Figure 1). The drugs were made by 15 different companies (range 1-3 drugs per company). The mean length of time between receipt of an NOC/c and fulfillment was 1390 (95% confidence interval 1160-1620) days.

![Figure 1](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/2/E295/F1.medium.gif)

[Figure 1](http://www.cmajopen.ca/content/5/2/E295/F1)

Figure 1 
Numbers of Notices of Compliance with conditions (NOC/c), Qualifying Notices and resultant publications.

Health Canada eventually supplied all the Qualifying Notices sought in the Access to Information Act requests, although in 1 case it took almost 20 months. Three Qualifying Notices (2 for different indications for amprenavir and 1 for nevirapine) had all the information about the confirmatory studies redacted, leaving the studies listed in 31 Qualifying Notices for 24 drugs made by 14 companies for analysis. These Qualifying Notices listed 1-7 confirmatory studies each. Health Canada sent only a single Qualifying Notice for delavirdine, although the product had received 2 separate NOCs/c and the letter from Health Canada acknowledged both NOCs/c. Therefore, I assumed that the 5 confirmatory studies in the Qualifying Notice for this product were the same for both NOCs/c. Two Qualifying Notices for 2 products (bortezomib and sunitinib) had 1 study in common. Thus, there were 58 distinct confirmatory studies that Health Canada required (Table 1).

View this table:
[Table 1:](http://www.cmajopen.ca/content/5/2/E295/T1)

Table 1:   Drugs with completed Notice of Compliance with conditions and number of confirmatory studies per drug

Descriptions of the confirmatory studies in the Qualifying Notices were highly variable, ranging from minimal (e.g., "final results of the pivotal Phase 3 study") to very detailed (e.g., "a randomised, double-blind, comparative, parallel-group, multicentre trial to evaluate the safety and efficacy of abacavir versus placebo in combination with background antiretroviral therapy in HIV-1 infected antiretroviral therapy experienced subjects with CD4+ cell counts > 100 cells/mm3 and plasma viral load between 400 copies/mL and 50 000 copies/mL"). At times, it was not clear whether the company was being asked to provide just a protocol for a study or the protocol and then the results; in these cases, I assumed that the companies were asked for the results of the studies. Some Qualifying Notices gave the trial number of the study, some did not, and in some cases it was redacted. Only 1 Qualifying Notice included a [clinicaltrials.gov](http://clinicaltrials.gov) identifier for 1 study. Similarly, only 1 Qualifying Notice listed a specific deadline for filing the results of the confirmatory studies.

Of the 14 companies contacted, 11 responded (17 products) and 3 did not respond (7 products) (Table 1). The companies that responded were responsible for 34 studies and identified a total of 29 unique publications that reported on 31 of these studies. For 2 products (gefitinib and sorafenib), a single publication reported on 2 studies. Finally, for 2 products (anastrozole and riluzole), studies were described in the Qualifying Notices, but the response from the companies did not address whether publications existed for some or all of the studies. No publications could be found for 1 study required for each product, and 1 publication was identified through database searches for anastrozole (Table 1).

The companies that did not respond were responsible for 24 of the studies. A total of 18 publications for 19 studies (1 publication reported on 2 studies of raltegravir) were identified through searches of [clinicaltrials.gov](http://clinicaltrials.gov), PubMed and Embase. No corresponding publication was found for 5 studies (Table 1).

If all the publications identified through database searches and not confirmed by the companies involved were correct, 51 (88%) of the 58 confirmatory studies were eventually published, as 48 full journal articles. Of the 48 articles, 31 (65%) (19 confirmed by companies and 12 identified through database searches) were published before the NOC/c was fulfilled, with a mean of 610 (95% confidence interval 439-780) days. Seventeen publications (35%) (11 confirmed by companies and 6 identified through database searches) appeared after the NOC/c was fulfilled, with a mean of 572 (95% confidence interval 338-805) days (Table 2).

View this table:
[Table 2:](http://www.cmajopen.ca/content/5/2/E295/T2)

Table 2:   Time between fulfillment of Notice of Compliance with conditions and publication of confirmatory study*

## Interpretation

Based on the assumption that the publications identified through database searches and not confirmed by the companies were correct, 88% of the confirmatory studies on efficacy/effectiveness were eventually published. Sixty-five percent of those published appeared a mean of 1.7 years before Health Canada declared that the NOC/c had been fulfilled. The remaining 35% of the publications appeared, on average, 1.6 years after NOC/c fulfillment. Up to 12% (7/58) of confirmatory studies may not be published at all, and in these cases the detailed information in the studies about the efficacy of the product will not be available to health care practitioners. The reasons for nonpublication are not clear. Since the studies were performed to fulfill the requirements of the NOC/c, it is highly likely that they were favourable, and therefore negative results are probably not an explanation. Companies may not have felt it necessary to submit the studies for publication since the drugs were already approved, or the studies may have been rejected by journals. Interestingly, 5 of the 7 confirmatory studies for which no publication was identified came from the 3 (out of 14) companies that did not respond to the request for information.

The average time from granting of an NOC/c to its fulfillment in the current study was 1390 days, or 3.8 years. The conditional approval mechanism of the European Medicines Agency also requires confirmatory studies. From the inception of this pathway, in 2006, until April 2014, 21 medicines requiring 59 confirmatory studies were approved; the studies were expected to take a median of 575 (interquartile range 204-1287) days (1.6 yr) to complete.6 Twenty-six of these studies were completed, but it took a median of 275 (interquartile range 121-773) days (0.8 yr) longer than expected.6 There are a number of possible explanations why completion of confirmatory studies may be different in the different jurisdictions. The type of studies required may be different (e.g., randomized controlled trials v. observational studies), the European Medicines Agency may be more diligent in monitoring the status of the studies, and the drugs requiring studies may be different; in fact, there were only 4 drugs in common between the 2 agencies. Hoekman and colleagues6 did not give the time to fulfill the marketing conditions for individual drugs, so times for these 4 could not be compared.

### Limitations

Three companies responsible for 24 of the 58 confirmatory studies failed to respond, and, therefore, there is uncertainty about whether the 18 publications for 19 of these studies were correctly identified, especially since in many cases confirmatory studies were often vaguely described, which makes it difficult to construct precise search strategies. In other cases, publications may have been missed, as the grey literature was not searched on the grounds that these publications would not have been readily available to health care practitioners.

### Conclusion

Most confirmatory studies were eventually published, but publication does not necessarily translate into access. Health care practitioners need to be provided with information to allow them to appropriately prescribe medications. To this end, when a drug is approved under an NOC/c, Health Canada and the manufacturer should jointly take responsibility for ensuring that clinicians are aware of the preliminary nature of the evidence for the product and the details of the confirmatory studies that are required. This could be accomplished through a detailed posting on the Health Canada Web site as well as sending "Dear Doctor" letters and requiring specific information about the nature of the uncertainty of the evidence in any promotional material. When the confirmatory studies are completed and accepted by Health Canada, Health Canada and the relevant company should use similar communication strategies to convey this information, along with the availability of the publication(s), to health care practitioners. These communication methods would need to be evaluated to be sure that they are achieving their objective.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/5/2/E295/suppl/DC1](http://www.cmajopen.ca/content/5/2/E295/suppl/DC1)

## Acknowledgements

Acknowledgement: The author thanks Catherine Oliver for the duplicate data extraction.

## Footnotes

*   **Competing interests:** In 2015-2016, Joel Lexchin received payment for being a consultant to a project looking at indication-based prescribing and a project that determined what drugs should be provided free of charge by general practitioners. He also received payment for being on a panel that discussed establishing a pharmacare program in Canada. He is on the Foundation Board of Health Action International.

*   **Contributors:** Joel Lexchin gathered and analyzed all the data for this study and wrote the manuscript. He approved the final version to be published and agreed to act as guarantor of the work.

## References

1.  (2002) Notice of compliance with conditions (NOC/c) [guidance document]. Ottawa: Health Canada.
    
    

2.  (2011) Notice of compliance with conditions (NOC/c) [guidance document]. Ottawa: Health Products and Food Branch, Health Canada.
    
    

3.  Lexchin J (2007) Notice of compliance with conditions: a policy in limbo. Healthc Policy 2:114–22.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19305737&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE295.atom) 

4.  (2012) Frequently asked questions: Summary Basis of Decision (SBD) Project - phase II. Ottawa: Health Canada. Available[www.hc-sc.gc.ca/dhp-mps/prodpharma/sbd-smd/sbd\_qa\_smd\_fq-eng.php](http://www.hc-sc.gc.ca/dhp-mps/prodpharma/sbd-smd/sbd_qa_smd_fq-eng.php). accessed 2014 Dec. 13.
    
    

5.  Law MR (2014) The characteristics and fulfillment of conditional prescription drug approvals in Canada. Health Policy 116:154–61.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.healthpol.2014.03.003&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24703857&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE295.atom) 

6.  Hoekman J, Klamer TT, Mantel-Teeuwisse AK, et al. (2016) Characteristics and follow-up of postmarketing studies of conditionally authorized medicines in the EU. Br J Clin Pharmacol 82:213–26.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/bcp.12940&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26992001&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE295.atom) 

*   Copyright 2017, Joule Inc. or its licensors