Published on in Vol 5, No 1 (2016): Jan-Mar

Surveying End-of-Life Medical Decisions in France: Evaluation of an Innovative Mixed-Mode Data Collection Strategy

Surveying End-of-Life Medical Decisions in France: Evaluation of an Innovative Mixed-Mode Data Collection Strategy

Surveying End-of-Life Medical Decisions in France: Evaluation of an Innovative Mixed-Mode Data Collection Strategy

Original Paper

1Institut National d'études Démographiques, Department of Survey and Sampling, Paris, France

2Institut National d'études Démographiques, Paris, France

3Vrije Universiteit Brussel, Head Department of Public Health, Brussels, Belgium

4Vrije Universiteit Brussel, End-of-Life Group, Brussels, Belgium

Corresponding Author:

Stephane Legleye, PhD

Institut National d'études Démographiques

Department of Survey and Sampling

133 Boulevard Davout

Paris, 75020


Phone: 33 1 56 06 20 98

Fax:33 1 56 06 21 92


Background: Monitoring medical decisions at the end of life has become an important issue in many societies. Built on previous European experiences, the survey and project Fin de Vie en France (“End of Life in France,” or EOLF) was conducted in 2010 to provide an overview of medical end-of-life decisions in France.

Objective: To describe the methodology of EOLF and evaluate the effects of design innovations on data quality.

Methods: EOLF used a mixed-mode data collection strategy (paper and Internet) along with follow-up campaigns that employed various contact modes (paper and telephone), all of which were gathered from various institutions (research team, hospital, and medical authorities at the regional level). A telephone nonresponse survey was also used. Through descriptive statistics and multivariate logistic regressions, these innovations were assessed in terms of their effects on the response rate, quality of the sample, and differences between Web-based and paper questionnaires.

Results: The participation rate was 40.0% (n=5217). The respondent sample was very close to the sampling frame. The Web-based questionnaires represented only 26.8% of the questionnaires, and the Web-based secured procedure led to limitations in data management. The follow-up campaigns had a strong effect on participation, especially for paper questionnaires. With higher participation rates (63.21% and 63.74%), the telephone follow-up and nonresponse surveys showed that only a very low proportion of physicians refused to participate because of the topic or the absence of financial incentive. A multivariate analysis showed that physicians who answered on the Internet reported less medication to hasten death, and that they more often took no medical decisions in the end-of-life process.

Conclusions: Varying contact modes is a useful strategy. Using a mixed-mode design is interesting, but selection and measurement effects must be studied further in this sensitive field.

Interact J Med Res 2016;5(1):e8



Improved living conditions, public health initiatives, and advances in medical care in most developed countries have contributed to a rise in life expectancies and a significant shift in the causes of death. Deaths from acute infectious diseases have declined, whereas deaths from chronic and degenerative diseases such as cancer and cardiovascular pathologies have increased [1,2]. More and more, people are dying at older ages, often in hospitals where they are permanently assisted by physicians and other health care workers who are directly involved in the dying process. These professionals can administer drugs to alleviate pain or other symptoms, and withhold or withdraw treatment to prolong life; however, the deliberate hastening of death is legally forbidden in France. Consequently, the quality of medical care at the end of life has become a significant concern in many societies, and it has become necessary to monitor related medical decisions reliably, to provide data to inform debates on this sensitive subject. Legislation concerning medical end-of-life decisions varies widely in Europe [3]. Over the last few decades, several European countries have performed single surveys or a series of surveys on this topic, including the Netherlands, Belgium, Denmark, Sweden, Switzerland, and Italy [4-8]. In France, major changes were introduced to the legislation in 2005 [9]. However, no nationwide representative survey has ever been undertaken, and the only available data has come from studies in hospitals and emergency wards [10-13]. Thus, there is a lack of knowledge of overall medical practices.

One of the main obstacles to conducting population-based studies is that health care professionals are recognized as a “hard-to-reach” population [14], resulting in high nonresponse rates. In combination with substantial differences between nonrespondents and respondents, this can lead to considerable nonresponse bias [15], which undermines the validity of these studies. For this reason, efforts aimed at improving participation rates must be concerned with every aspect of a protocol [16-18]. First, positive impacts have been noted using the following methods: varying contact modes and/or the form of postal contact (letters, cards, telephone, etc) [17], personalized cover letters, replacement questionnaires combined with a high number of follow-up contacts, and a long data collection period [19,20]. Second, mixing data collection modes (eg, mail and internet) may show positive results, although postal questionnaires are often favored over telephone and Web-based surveys [21,22]. Finally, surveying a sample of nonrespondents may also improve data quality, through the assessment of reasons for refusal and/or the determination of sociodemographic or professional characteristics of nonrespondents [17,23]. This latter technique was used in the 2001 end-of-life decision surveys in the Netherlands, Switzerland, and Sweden [24], as well as in the 2007 Belgium survey [25].

The survey Fin de Vie en France (“End of Life in France,” abbreviated as EOLF) was conducted in 2010 by the Institut national d’études démographiques (INED), with the purpose of describing end-of-life medical decisions in the French context [26]. Compared to previous surveys, EOLF comprised several innovations, including a mixed-mode procedure (internet and postal questionnaire) as well as postal and telephone follow-ups, combined with postal or email reminders sent by the medical authorities (hospitals and regional health agencies). It also comprised a nonrespondent telephone survey to assess nonresponse bias.

The aim of this paper was to describe and evaluate the methodological innovations of EOLF and to assess their impact on data collection quality. It describes response rates, representativeness of the sample, motives for nonresponse, and differences resulting from the data collection modes. Regarding the mixed-mode methodology, we assessed whether the choice of Internet over paper questionnaires was linked to the characteristics of the participating physician or of the deceased person, and whether this choice had an impact on the reporting of end-of-life medical decisions.

Retrospective Design

We chose to sample deaths and not physicians [4-8] for the same reasons given by Chambaere et al [25]. A representative sample of 14,999 deaths was selected by the CepiDc (Centre d'épidémiologie sur les causes médicales de décès) de l’Institut National de la Santé et de la Recherche Médicale (INSERM), and was drawn using systematic random sampling (sorted by age, gender, place of death, and region of residence of the deceased person [27]) from among 47,872 deaths of persons aged 18 and over that occurred in continental France in December 2009.

The certifying physician was identified, and a questionnaire about the selected deceased person(s) was mailed to her/him with instructions for replying. Physicians could respond either by post or by using a specially developed secure Web-based questionnaire. When physicians had more than one death in the sample, for each death, we provided an identifier and password to use in the Web-based mode and a questionnaire with a prepaid envelope to reply by post.

Anonymity and Follow-Up

While the Belgian survey procedure employed a lawyer as a third party, preliminary discussions in France concluded that any mention of a lawyer for this kind of survey would provoke reluctance rather than reassure physicians to participate. We used the services of a specialized hospital department to play the role of the trusted third party for the paper administration [28]; the members of this service also entered the paper-based questionnaire responses manually using the secure Web-based questionnaire that was developed.

The Web-based administration adapted the same approach by involving a trusted virtual party [29]. This method used more than one Web server; answers entered by the practitioner were neither sent to nor temporarily saved on the first Web server (the one to which they logged on). Instead, the responses were sent to a second server using .xml files, for which filenames included a second and different irreversible hash chain of the death identity. In addition, access to log files was not activated on this host Web server, in order to suppress information regarding Internet protocol addresses. Furthermore, no electronic acknowledgment was sent to physicians, and all .xml files received had the same date of creation on the host server. This overprotection of the practitioner’s identity contrasts with current philosophy of Web security, which consists of tracking connections to a server and forcing browser clients to dialog with a unique, well-identified server. The material sent to physicians explained how their anonymity was ensured.

No link could be established between the physician and the information in the death certificate (see Questionnaires and Data Quality Check).


Contact Material

Following the recommendations of Dillman [17], the first mailing was personalized (name and address of the physician), and the leading scientific institutions were clearly identified on the envelope. It contained: an invitation letter, which was signed by the heads of the leading scientific institutions involved and which also explained the importance of the study; a follow-up card for cases in which the physician who had only certified the death could fill in the name and address of the physician who had actually treated the patient until her/his death; a color-printed questionnaire; and a prepaid envelope for returning the questionnaire or card. A leaflet was also provided, which presented the survey and a flyer describing how the anonymity of respondents was ensured. For anonymity purposes, the completed questionnaire was placed in a sealed envelope inside the prepaid envelope (as for absentee postal voting).

The survey was also advertised by the main French medical bodies/authorities and by regional health agencies before and during data collection, as well as in a press release issued just before the survey went out to the field.


The questionnaire was 20 pages long in A5 booklet form, with only closed questions and no space for writing any comments, as required by the French data protection committee (CNIL). The questionnaire of around 100 questions comprised the following sections: characteristics of the responding physician; characteristics of the deceased person; place of death; treatments (palliative care, sedation, etc); information related to the 2005 law (health-care proxy/surrogate, living will or advanced medical directives, discussion of last medical decision with patient or surrogate, and discussion with colleagues/collegiality of discussion); medical end-of-life decisions; wish of the deceased person to hasten death and/or euthanasia; medical practitioners or nursing staff involved in the end-of-life period; and visits of family/friends [30].

Although the age and sex of the deceased person, region of death, and month of death were identified on the death certificate and sent to the physician to identify the death, we asked for some of this information in the questionnaire because, in order to preserve anonymity, questionnaires were not merged with death certificates.

The questionnaires used in the phone-call campaign and in the nonresponse survey (which was also administered by phone) were shorter than the main questionnaire. Although these questionnaires focused on the motives for nonresponse, they also included several questions identical to those in the main questionnaire, for describing the physicians’ characteristics.

Data Collection

Identification of the Certifying Physician

Names of physicians and their professional addresses were identified by their signatures and stamps on the death certificates and entered manually. In ambiguous or unreadable cases, names and addresses were requested from the mayor’s office that recorded the certificates. The names and addresses of each physician were checked manually using the Internet and the administrative register of all physicians in France. The result of this preliminary phase is presented in Table 1.

Table 1. Identification of physicians/death certificates.a
Samplen or %
Initial sample of death certificates (1)14,999
Death certificates with directly identified physician (2)11,412
Death certificates with unidentified physician, sent to mayor’s offices (3)3587
Physician identified by the mayor’s offices (4)2828
Impossible to identify the physician (5.1)759
Certificate disregarded due to the cut-off of 4 certificates per physician, except in cases of heads of department (5.2)160
Final sample, death certificates (6)=(2)+(4)−(5.2)14,080
Number of physicians11,828
Proportion of unused death certificates (10)=[(5.1)+(5.2)]/(1)6.12
Proportion of used death certificates (11)=(6)/(1)93.87

aThe figures in parentheses are useful to compute % in this table.

Eligibility of the Physician

The major reasons for not including a physician in the study were the inability to find a professional workplace address or a late reply from the mayor’s office (a cut-off point was set for the end of May, and we discarded late returns from the Mayor’s office). To reduce refusal due to workload, we set a limit of four questionnaires per physician. An exception was made for department heads in institutions, because we discovered during the pilot survey that some heads of departments signed the certificate for most, if not all, deaths that occurred in their department (often treated by another physician). Thus, we anticipated that department heads would distribute questionnaires to corresponding treating physicians.

Fieldwork Stages

Data collection comprised three stages. The first stage was a postal phase. The initial mailing, including the questionnaire, was sent on 2010 May 25. A first postal reminder letter was sent to the nonrespondents two weeks later. Two weeks after that reminder, a third mailing was sent with the same material as the first one, but with adapted text. We avoided sending letters during the summer holiday period. A fourth and last reminder was sent to nonrespondents in September. The entire procedure was enhanced in three ways. First, there was a press release issued just before the survey. Second, we sent a letter in July to all directors of hospitals with at least one physician who had been selected for our sample, to ask him/her to convince their physicians to participate. Third, regional health agencies sent a similar motivational letter to institutions during the second half of August, asking them to communicate with physicians about the study and requesting that they pay attention to it and participate if they had been selected.

The second stage of data collection began on 17 Jun 2010 with a telephone campaign. At that time, there were 10,677 nonrespondent physicians, and a telephone number was known for 10,582 of them. The aim was to personally contact 7,000 of these physicians by phone to convince them to participate or, if they declared that they did not want to participate in the survey, to report their motives for refusal (for financial reasons, we could not contact all physicians). For this purpose, a sample of 9210 nonrespondents was drawn from among the 10,677 nonrespondents at the beginning of the telephone campaign (86.26%). This sample comprised all physicians who had signed at least two death certificates (n=1565), and others who were included randomly. Each number was to be contacted up to 16 times before being abandoned (globally, the mean number of contact attempts was 4.8). This number of call attempts was chosen based on efficiency and budgetary considerations. In parallel, a third letter, similar to the second one, was sent to 95 nonrespondent physicians whose phone number was unknown. This stage took place in two phases: 17 June-21 July, and 4 October-5 November.

The third stage was a telephone survey of 1080 final nonrespondent physicians. It began on 7 December 2010 and ended on 7 January 2011. Each phone number was to be contacted up to 20 times before being abandoned. The goal of this survey was only to collect the motives for nonresponse, along with basic characteristics of the physicians (age group, sex, and specialty). To obtain an accurate measure of these, we set a high number of call attempts because we anticipated that these physicians would be very hard to contact (Figure 1 shows the data collection modes).

During all fieldwork, a hotline (8 am-8 pm) was offered to provide information about the survey and to resend materials to physicians in case of loss.

Figure 1. Schedule of data collection fieldwork and number of questionnaires by data collection mode.
View this figure

Data Quality Check

We checked that the month of death reported by physicians in the questionnaire was December 2009, to be sure that the deaths they were reporting on were those selected in the sample and not arbitrary ones (such as the most recent or a more interesting case). Previous end-of-life decision surveys in other countries did not take this precaution.


The final respondent sample was weighted using a calibration procedure [31], considering age × sex, and region and place of death, as observed in the initial sample of deaths.

Ethical Considerations

This survey was approved by the Comité Consultatif sur le Traitement de l’Information en matière de Recherche dans le domaine de la Santé (CCTIRS) in January 2010 and authorized by CNIL (authorization number 1410166).

Statistical Analysis

Samples were described using percentages and bivariate analyses with Pearson chi-square tests. Three multivariate logistic models (providing adjusted odds ratios and 95% confidence intervals) were also computed when comparing Web-based and paper questionnaires. These models tested whether the choice of Web-based questionnaires was linked to physician characteristics (Model 1), death characteristics (Model 2), or both sets of characteristics (Model 3). All statistics were computed using SAS V9.3 and were nonweighted unless specified.

Preliminary Identification of Death Certificates, Physicians, and Participation Rate

Overall, 14,080 death certificates (93.87% of the initial sample of deaths) with identified physicians were available for the survey, corresponding to 11,828 different physicians (Table 1). The final sample was reduced to 13,460 deaths because of postal address problems (changes in professional location, etc). From this sample, 5217 questionnaires were completed and returned. This led to a participation rate of 40.02% [32].

We used the standard Response Rate 2 from the American Association for Public Opinion Research (AAPOR). The formula used was 5217 questionnaires/(5217 questionnaires + 1506 refusals + 449 letters not delivered + 561 physicians who could not respond because the survey did not concern them [sudden death, not the physician in charge of the patient, could not remember the case or could not find the file] + 49 other reasons for nonresponse [eg, death of respondent, retirement, not available during data collection] + E[6287 neither responding nor refusing]) = 40.02%. E is the estimation of the proportion of eligible cases (in this case, 92.10%). E was determined by the ratio of the sum of questionnaires + refusals + others, to the sum of questionnaires + refusals + others + non-eligible persons. We counted as non-eligible those who mentioned in the follow-up survey that they did not respond because the survey did not concern them (not in charge of the patient, not a forensic scientist, could not remember or find the file of the person, etc).

An additional analysis of the follow-up file (not mentioned in the table) showed that the response rate varied with the patients’ characteristics. It varied by age of the deceased (from 42.12% for deceased individuals aged 18-39 to 35.03% for those aged 90+), place of death (from 29.15% for nursing homes to 17.17% for public places and 40.20% for public hospitals), and region of residence (from 30.76% in the Mediterranean region to 40.42% in the East). It also varied with the physicians’ characteristics: participation of women was higher (43.25% vs 34.14%), and type of practice exhibited variation (from 30.92% for those working in private practice—regardless of whether they were general practitioners or specialists—to 35.67% for those in emergency services and 39.12% for those working in other services at a public hospital).

Data Quality Check

During the data quality check, 311 questionnaires were discarded because either the reported month of death was not December (230) or the month of death was missing (81). Chi-square tests showed that these 311 questionnaires were slightly different for some medical decisions (treatment prolonging life: 35.37% vs 31.92%, P=.26; withdrawal from treatment: 16.10% vs 21.81%, P=.0041; treatment to alleviate pain: 44.37% vs 40.85%, P=.221; and medication to hasten death: 2.25% vs 0.74%, P=.004). Another 15 questionnaires could not be used because of computer problems encountered by the physicians. We thus decided to exclude all these deaths from the analysis: the final sample contained 4891 valid questionnaires.

Sample Structure

Table 2 exhibits the nonweighted sociodemographic structure of the initial sample drawn from the national death register and of the final death sample that was analyzed. This structure was close to the actual structure of all deaths in December, except for a small overrepresentation of deaths in public hospitals.

Telephone Campaign

The second stage of data collection was a telephone campaign: 6169 physicians/secretariats were contacted from the sample of 9220 nonrespondents at that time. Finally, 5421 physicians were spoken to personally and participated in the telephone campaign (Table 3).

Among these 5421, 505 (9.32%) felt the EOLF survey did not concern them (and thus the telephone campaign), 1106 (20.38%) reported that they were currently participating in the main survey, 2621 (48.35%) stated that they wanted to participate in the EOLF survey, while 1189 (21.93%) explicitly refused to participate. Of the physicians who wanted to participate, 1098 (41.89%) were not aware of the survey, 542 (20.67%) asked for the material to be sent again, and 981 (37.43%) promised to participate soon. The response rate to the telephone campaign was thus 63.21% (following the AAPOR standard Response Rate 2). Among the 1189 refusing physicians, 332 (27.92%) indicated that they never participate in surveys, 603 (50.71%) said that they were currently too busy, and 36 (3.03%) mentioned the absence of a financial incentive. In addition, 100 (8.41%) mentioned reasons related to the usefulness of such a survey, 29 (2.44%) reported that the subject was too personal and/or too sensitive, and another 25 (2.10%) reported other reasons related to the methodology of the survey. The other 20 refusing physicians did not specify their reasons (1.68%). Overall, the telephone campaign seemed to strongly improve the participation rate, as shown in Figure 1. The effects of the other mailings and follow-up interventions of the hospitals and health agencies were less clear, although they may also have contributed to the success of the phone campaign. The effects of the telephone campaigns on the Web-based questionnaires appeared modest compared to those on the paper questionnaires: the number of collected Web-based questionnaires was almost stable after the first phone follow-up.

Table 2. Sociodemographic characteristics of the sample of deaths: initial sample, selected sample, and returned questionnaires (%).

Initial sample
Selected sample
(after removals)
Final sample/analysis filea (nonweighted)
All deaths in Decemberb




18-49 years7755.177125.062905.8825855.39

50-69 years279018.60110518.3494619.34910618.98

70-79 years300720.05283520.1398020.04947819.75

80-89 years578538.57543938.63183937.6018,44338.43

≥90 years264217.61251217.8483617.09837417.45
Place of patient death


Public hospital741549.44696549.47270555.3123,46048.89

Private hospital13348.8912298.733487.1240158.37

Nursing home231015.40223715.8967313.76562511.72

Public place1741.161591.13821.686431.34


Region of death

Ile de France200513.37184713.1262912.85643413.41

Bassin Parisien267917.86253518.0091918.79877718.28







aAs reported on the questionnaire by the physician.

bBased on all coded death certificate data from December.

Table 3. Results of the telephone campaign and motives for nonresponse.a

Contact achieved


Physician personally contacted and participated in the telephone campaign

Phone call blocked by the secretariat

Contact made but physician not reachable
No contact achieved

Personally contacted physicians who participated in the telephone campaign


Explicitly refused to participate in the main survey

Did not concern themb

Participation in the main survey was in progress

Already sent the questionnaire5069.33

Transferred the material to the right physician60011.07

Wanted to participate in the main survey

Was not aware of the survey109820.25

Asked for the material to be sent again54210.00

Promised to participate soon98118.10
Motives for explicit refusals to participate in the main survey (multiple responses)


Lack of time

Never participate in surveys

Too many surveys

Absence of financial incentive

Questionnaire too long

Reason related to survey usefulness

Reason related to survey topic

Reason related to survey methodology

Not specified

aInitial nonresponse sample (N) = 9220.

bForensic scientists are the certifying physician in cases of violent or suspicious deaths.

Survey of Nonrespondents

The survey of the final nonrespondents, conducted by phone from December 2010 to January 2011 (the third stage of data collection), used a random sample of 1080 physicians with valid phone numbers who did not express explicit refusals but also did not respond during fieldwork. Contact was made in 957 cases, and 684 physicians could be contacted personally (Table 4). Finally, 547 physicians agreed to participate in the nonrespondent survey (79.97% of all personally contacted physicians); 38 reported that they had already participated in the main survey and 52 said that they would still participate, leading to a response rate of 63.74% (as above, this rate is computed following the AAPOR standard Response Rate 2).

Among respondents, the most frequent motive for nonresponse to the main survey was lack of time (53.02%). However, some survey-specific motives were frequently reported: it was impossible to remember the deceased person, or it was too difficult to find the medical file (14.26% and 9.69%). Almost 6% (5.85%) of the physicians reported having been unaware of the survey, indicating difficulties in contacting them personally.

Table 4. Results of the telephone nonrespondent survey (third stage) and motives for nonresponse (N=1080).

No answer obtained


Always get the answering machine

No answer after 20 call-backs

Number not valid

Other reasons
Answer obtained


Physician personally contacted

Participation in the nonrespondent surveya54750.65

Refusal to participate in the nonrespondent survey474.35

Already answered the main survey383.52

Not aware, will participate in the main survey524.81

Refusal from secretariat

Refusal from the physician

Impossible, not eligible
Motives for nonresponse (multiple responses)b


Lack of time

Does not remember the case

Was not in charge of the deceased person

Forgot to answer/lost the questionnaire

Too difficult to find the medical file

Unaware of the survey

Questionnaire too long

Reason related to the survey topic

Reason related to the survey methodology

aA total of 78 physicians who answered the nonresponse survey mentioned that they had already answered the main survey or that they would do so. bThe percentages do not add to 100 because this was a multiple-response question.

Differences Between Internet and Paper Responses

Overall, 73.21% of the questionnaires (n=3557) were on paper while 26.78% (n=1334) were collected through the secure Web-based questionnaire. As shown in Table 5, physicians who chose the Internet were more often male, younger, and working in large towns or institutions. Compared to general practitioners (GPs) in private practice, almost all specialists were more likely to choose the Internet, especially anesthesiologists and GPs in hospitals (but not oncologists or cardiologists). Causes of death were not exactly similar in both modes (P=.033): cancer was more frequent for the paper questionnaire (28.45% vs 25.19%, P=.019) and infectious diseases more frequent for the Web-based questionnaire (8.80% vs 6.28%, P=.003). Medical decisions were similarly distributed in the two samples (P=.114), although medications to hasten death were more frequent for the paper questionnaire (0.96% vs 0.33%, P=.036, crude odds ratio [OR] 0.35, 95% CI 0.13-0.96).

To test whether these bivariate differences could be explained by patient or physician characteristics, we ran three logistic models. Controlling for physician characteristics, Model 1 shows that physician differences between the Web-based and paper questionnaires remained, and that reporting “no decision” was more frequent for Web-based questionnaires (OR 1.44, 95% CI 1.08-1.71).

Table 5. Comparison of questionnaires completed on the Internet versus on paper: percentages, adjusted odds ratios (OR), and 95% confidence intervals (95% CI).
CharacteristicsInternet sample
Paper sample

Physician characteristic
(Model 1)
Patients characteristic
(Model 2)
Physician and patient characteristics
(Model 3)

%%PbOR95% CIOR95% CIOR95% CI
Physician gender




Physician age


18-39 years old29.2021.40







Physician medical specialty


GPs in private practice11.2120.69






Geriatrists in hospitals6.577.56


Geriatrists elsewhere10.9211.91


Emergency physicians17.7813.54




Other specialist in hospital3.743.80


Other specialist outside hospital11.8712.50


Other GPs7.368.32


GPs in hospitals5.715.40

Physician town size











Death certificates (3 months)













Physician medical decision


Sudden death17.5316.68

Life-prolonging treatment12.6212.04

Treatment withheld13.1215.10

Treatment withdrawn4.943.96

Intensity of pain alleviation27.4628.31

Medication to hasten death0.330.96

None of the above24.0022.95
Patient gender





Patient age











Place of death






Nursing/retirement home12.0316.69



Cause of patient death




Cardiovascular disease24.4924.32.900


Neurological disease13.9114.90.362


Infectious disease8.806.28.003


Respiratory disease6.766.52.729


Digestive disease5.004.03.113


Mental health3.013.12.819


Violent death3.073.59.417




Controlling for patient characteristics, Model 2 shows that deaths of people aged 90 and over were less often reported using the Web-based questionnaire, as were deaths outside the hospital. Furthermore, compared to cancer, most other causes of death (except violent or sudden death) were reported more often in the Web-based questionnaire. The administration of medication to hasten death was reported less in the Web-based questionnaire (OR 0.35, 95% CI 0.12-0.99).

When both physician and patient characteristics were controlled (Model 3), only physician characteristics were significantly associated with response mode. Regarding end-of-life decisions, reporting “no decision” was more prevalent for the Web-based questionnaire (OR 1.39, 95% CI 1.09-1.78), as in Model 1.

In all three models, medication to hasten death tended to be reported less through the Web-based questionnaire (although, significantly, only in two models). Thus, choosing the Web-based questionnaire was associated mainly with physician characteristics, but Web-based questionnaires still presented specificities compared to paper questionnaires: reports of medications to hasten death were rarer, whereas those of “no medical decision” were more frequent.

Principal Findings

This is the first mixed-mode survey on end-of-life medical decisions. It followed most of the methodological recommendations in the literature for improving response rates [17-19]. Phone calls and postal reminders had a strong impact on participation (Figure 1), and all of these efforts contributed to a robust sample of respondents, despite a modest response rate (40.02%).

Female physicians were more likely to respond than males, as were physicians working in hospitals compared to those in private practice, while paper questionnaires were favored over Web-based questionnaires. Nevertheless, Web-based questionnaires were favored by physicians with certain characteristics: males, specialists, those who were younger, those working in neurology, emergency, and geriatrics, and those in large towns and institutions. Reports of medication to hasten death were also rarer for Web-based questionnaires (0.33% vs 0.96%), while the multivariate results suggested that there might have been fewer reports of illegal decisions through Web-based questionnaires, even when adjusting for physician characteristics.

The following paragraphs focus on the interpretation of these results and provide comments on the efficacy and drawbacks of our survey protocol.

Mixed-Mode Methodology

As our mixed-mode protocol used only self-administered questionnaires, the effect of the data collection mode on results may have been limited. In the case in which the selection of the data collection mode is controlled, data quality may be slightly higher by Internet [33] due to higher internal consistency. We wondered if this was the case and concluded first that the paper questionnaire was favored over the Web-based questionnaire, as found in previous studies in Australia, the United States, and Canada [21,22,34]. Secondly, the fact that some types of physicians favored the Web-based questionnaire was in line with the literature about mixed-mode surveys in the general population (young males living in large cities are more likely to respond by Internet); however, others were specific to the working conditions of the physicians (working in an institution instead of private practice), while others merit further investigation, such as medical specialty (neurology and emergency). Multivariate regressions showed that answering on the Internet was mainly associated with physicians’ characteristics, but Web-based questionnaires still presented some specificities: medication to hasten death was more rarely reported and the absence of a medical decision was more frequently reported. Thus, Web-based responses seem to concern less problematic (controversial or illegal) decisions. The choice of the Internet may reflect a selection effect related to the medical practice and typical type of medical decision made by the physician or a deliberate choice of the Internet for less problematic deaths. On the other hand, it may be a true data collection mode effect; for example, a physician who responds on the Internet may be more reluctant to declare controversial or illegal decisions, regardless of the type of death or medical decision. With the absence of randomization and nonresponse, disentangling the two is impossible. Because of the specificity of the respondents and the topics covered in the most sensitive questions, we cannot determine the direction and magnitude of the bias that occurred for each data collection.

We also noticed that the effects of the follow-up interventions (phone calls and postal reminders) were lower for Web-based questionnaires than for postal questionnaires. This is also in line with the literature on Web-based surveys, which shows that if people do not participate immediately, they tend to feel less concerned about a survey within a short time after they have been contacted. It may also be due to the absence of any email reminders.

According to Scott et al [35], using a simultaneous mixed mode may not be the most efficient protocol for surveying physicians: it is better than using only a Web-based survey, but it is costlier. Furthermore, it provides no further benefits than using a sequential mixed-mode that begins on the Internet. Unfortunately, the opposite sequence was not compared. It is likely that the mixed-mode increased the participation rate compared to either a Web-based or paper only survey, but we cannot measure to what extent. In our protocol, the reduction in costs was not substantial compared to a pure paper survey, because all contacts were made by post and only a small number chose to answer by the Internet.

Identification of the Deceased Person

We found that 311 questionnaires were related to a different month of death than the one expected. The reasons for this may be: (1) physicians reported on the month they participated in the survey rather than the month the person died; (2) they did not understand or see that the questions were about a specific case; (3) they did not have access to the person’s file but still wanted to participate, so they chose another case; (4) they wanted to respond about a specific case that—from their point of view—was more interesting. The significant differences between retained and discarded questionnaires suggest that, in some cases, these physicians may indeed have purposely chosen what they believed to be a “more interesting case.”

Comparison With Other Studies

The methodology of our survey was similar to the one applied by earlier nationwide epidemiological postal surveys in Europe [4,25]; however, it used a mixed-mode strategy with telephone call backs and postal reminders from different medical authorities. The trusted party was not allowed or able to link the characteristics of the deceased to the questionnaire sent by the physician, but we showed that asking for some characteristics of the deceased person allowed us to check for possible errors in their identity, which contrasts with previous Belgian surveys [8,36]. We did not use a lawyer as a third party: a preliminary discussion in France concluded that this choice would cause physicians to be reluctant and thus undermine the confidentiality and anonymity that we sought. The use of a specialized hospital department was well received (did not provoke any comment).

Our response rate was modest but comparable to what has been commonly found in other surveys among physicians in France [37]. Furthermore, we attained almost twice the response rate that was recently reported in a representative survey of all practitioners in Australia [35]. However, ours was lower than those of previous surveys of the same topic in other countries [4,8,36]. One reason for this low rate may be the fact that a large proportion of the physicians felt unconcerned, because they thought the deaths they were in charge of were rather ordinary (sudden death or death without any particular end-of-life decision).

Limitations and Recommendations for Future Surveys (in France and in Other Countries)

An apparent weakness is that we did not provide any incentive to improve our response rate. Prior research has demonstrated that prepaid monetary incentives (rather than nonmonetary) were effective in promoting survey participation [38-40]. In France, monetary incentives for surveys conducted by public institutions are uncommon, as opposed to studies sponsored by the pharmaceutical industry. As a consequence, no methodological evaluation study has been published on this topic. However, the success of financial incentive is not guaranteed: in a recent national telephone survey of GPs carried out by the National Institute of Health Education and Prevention in France, many participants refused the €30 incentive (equivalent to 1.5 times the consultation fee) to participate [41], arguing that this proved the survey may have had a commercial purpose. In the context of the EOLF survey, it might have been perceived as inappropriate. Nonmonetary incentives [39,40] may also improve participation, but it is not guaranteed, as demonstrated by a randomized, controlled experiment conducted among physicians in France [37]. In EOLF, 3.03% of refusals were explicitly linked to the absence of financial incentive; even if underreported, the effect on the participation rate was most likely marginal.

The fact that 19% of the contacted physicians in the call-back phase and 4.81% of the physicians in the nonrespondent survey were not aware of the original survey suggests that a phone call is necessary to overcome postal mailing problems, and we strongly recommend it for future surveys. For the same reason, although we could not assess the performance of this procedure, we recommend implementing public campaigns and reminders by medical authorities.

For data protection purposes, we made sure that no linkage was possible. This has three important consequences. (1) Due to the anonymity process, we were unable to eliminate potential paper and Internet questionnaires related to the same deceased person (if any). Some of the physicians used the same logins and passwords for all of their questionnaires and therefore generated tedious work for reconstituting each case. (2) The weighting process of the survey had to be simple, as no detailed information merging the initial sample and the respondent sample could be used, except when using aggregated data. (3) It is impossible to compute an accurate response rate by physician characteristics, because we could not merge the information in the sampling frame to that in the contact files and questionnaires. Future surveys must take these constraints into account. Our results nevertheless show that a good option for ensuring consistency in the questionnaire is to ask for broad characteristics of the deceased instead of linking them from the death certificate.

With a clear preference for the paper questionnaire and differences in reporting some important medical decisions, it seems too early to consider only an Internet-based survey in France. As precluding the internet does not appear to be an option currently—especially among young physicians—and as the topic and respondents of surveys on end-of-life medical decisions are very specific, we recommend conducting a mixed mode survey. Nevertheless, future research is needed for defining the best protocol (simultaneous vs sequential paper and Web-based), as well as for controlling selection and measurement effects in the data collection mode.


This paper is dedicated to the memory of Martine Quaglia, who supervised the fieldwork. This survey could not have been completed without the financial support of INED and the Health Ministry’s Direction Générale de la Santé (represented by A. Fontaine and E. Gaillard). We would like to thank the following individuals: The survey’s steering committee for their support and constructive discussions throughout the design and first analysis of the survey (Piernick Cressard and Francois Stefani of the Conseil National de l’Ordre des Médecins, National Medical Council; Eric Jougla, Albertine Aouba, and Grégoire Rey of CepiDc at Institut National de la Santé et de la Recherche médicale - French National Institute of Health and Medical Research; Chantal Cases, Director of INED for her constant support throughout the survey; Our “trusted third party” partners: Jeanne Fresson of the Department of Medical Information at the Maternité Universitaire de Nancy and Epiconcept for the Internet response channel; Régis Aubry (Physician, Head of the “Observatoire National de la Fin de vie”) and Silvia Pontone (Physician); from the Department of Survey and Polls at INED: Raphaël Laurent, Efi Markou, Martine Quaglia, Nicolas Razafindratisma, and Patricia Thauvin. And of course, all our thanks and gratitude to all the physicians who gave their time to take part in the survey.

Authors' Contributions

SL created the first draft and participated in the survey design. SP defined the survey design, analyzed the data, and made critical revisions to the manuscript. NB defined the pilot survey design. AS participated in the survey management and computed statistics. JB and JC made critical revisions to the manuscript. All authors read and approved the final manuscript.

Conflicts of Interest

None declared.

  1. Vallin J, Meslé F. De la transition épidémiologique à la transition sanitaire: L'improbable convergence générale. In: Tabutin D, Masquelier B, editors. Ralentissements, Résistances et Ruptures dans les Transitions Démographiques (Coll. Actes de la Chaire Quetelet 2010). Leuven, Belgium: Presses Universitaires de Louvain; 2013:257-290.
  2. Mazuy M, Prioux F, Barbieri M. Recent demographic developments in France. Some differences between the overseas departments and metropolitan France. Population-E 2011;66(3-4):423-472.
  3. Commission de réflexion sur la fin de vie. Rapport à François Hollande, Président de la République. France: Palais de l'Elysée; 2012 Dec 18. Penser solidairement la fin de vie   URL: http:/​/www.​​IMG/​pdf/​Rapport-de-la-commission-de-reflexion-sur-la-fin-de-vie-en-France.​pdf [accessed 2015-11-03] [WebCite Cache]
  4. van Delden JJ, Löfmark R, Deliens L, Bosshard G, Norup M, Cecioni R, et al. Do-not-resuscitate decisions in six European countries. Crit Care Med 2006 Jun;34(6):1686-1690. [CrossRef] [Medline]
  5. Giannini A. End-of-life practices in European intensive care units. JAMA 2003 Dec 10;290(22):2939-2940. [CrossRef] [Medline]
  6. Seale C. Counting end of life decisions. BMJ 2009 Sep 14;339:b3690. [CrossRef] [Medline]
  7. Deliens L, Mortier F, Bilsen J, Cosyns M, Vander SR, Vanoverloop J, et al. End-of-life decisions in medical practice in Flanders, Belgium: A nationwide survey. Lancet 2000 Nov 25;356(9244):1806-1811. [Medline]
  8. van der Heide A, Deliens L, Faisst K, Nilstun T, Norup M, Paci E, et al. End-of-life decision-making in six European countries: Descriptive study. Lancet 2003 Aug 2;362(9381):345-350. [CrossRef] [Medline]
  9. La République Française. Journal Officiel de la République Française. Paris; 2005. Loi n° 2005-370 du 22 avril 2005 relative aux droits des malades et à la fin de vie   URL: [accessed 2015-11-03] [WebCite Cache]
  10. Le Conte P, Riochet D, Batard E, Volteau C, Giraudeau B, Arnaudet I, et al. Death in emergency departments: A multicenter cross-sectional survey with analysis of withholding and withdrawing life support. Intensive Care Med 2010 May;36(5):765-772. [CrossRef] [Medline]
  11. Ferrand E, Dreyfus J, Chastrusse M, Ellien F, Lemaire F, Fischler M. Evolution of requests to hasten death among patients managed by palliative care teams in France: A multicentre cross-sectional survey (DemandE). Eur J Cancer 2012 Feb;48(3):368-376. [CrossRef] [Medline]
  12. Ferrand E, Robert R, Ingrand P, Lemaire F, French LG. Withholding and withdrawal of life support in intensive-care units in France: A prospective survey. French LATAREA Group. Lancet 2001 Jan 6;357(9249):9-14. [Medline]
  13. Ferrand E, Marty J, French LG. Prehospital withholding and withdrawal of life-sustaining treatments. The French LATASAMU survey. Intensive Care Med 2006 Oct;32(10):1498-1505. [CrossRef] [Medline]
  14. Field TS, Cadoret CA, Brown ML, Ford M, Greene SM, Hill D, et al. Surveying physicians: Do components of the “Total Design Approach” to optimizing survey response rates apply to physicians? Med Care 2002 Jul;40(7):596-605. [Medline]
  15. Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly 2006 Jan 01;70(5):646-675. [CrossRef]
  16. Dillman DA. Mail and Telephone Surveys: The Total Design Method. New York: Wiley Inter-Science; 1978.
  17. Dillman D. Mail and Internet Surveys: The Tailored Design Method, 2nd edition. New York: Wiley; 2007.
  18. Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009 Jul 08(3):MR000008. [CrossRef] [Medline]
  19. Parsons JA, Warnecke RB, Czaja RF, Barnsley J, Kaluzny A. Factors associated with response rates in a national survey of primary care physicians. Evaluation Review 1994 Dec 01;18(6):756-766. [CrossRef]
  20. Creavin ST, Creavin AL, Mallen CD. Do GPs respond to postal questionnaire surveys? A comprehensive review of primary care literature. Fam Pract 2011 Aug;28(4):461-467 [FREE Full text] [CrossRef] [Medline]
  21. Bonevski B, Magin P, Horton G, Foster M, Girgis A. Response rates in GP surveys - trialling two recruitment strategies. Aust Fam Physician 2011 Jun;40(6):427-430 [FREE Full text] [Medline]
  22. Nicholls K, Chapman K, Shaw T, Perkins A, Sullivan MM, Crutchfield S, et al. Enhancing response rates in physician surveys: The limited utility of electronic options. Health Serv Res 2011 Oct;46(5):1675-1682 [FREE Full text] [CrossRef] [Medline]
  23. Hansen, MH, Hurwitz, WN. The problem of non-response in sample surveys. J Am Stat Assoc 1946 Dec;41(236):517-529. [Medline]
  24. Miccinesi G, Fischer S, Paci E, Onwuteaka-Philipsen BD, Cartwright C, van der Heide A, et al. Physicians' attitudes towards end-of-life decisions: A comparison between seven countries. Soc Sci Med 2005 May;60(9):1961-1974. [CrossRef] [Medline]
  25. Chambaere K, Bilsen J, Cohen J, Pousset G, Onwuteaka-Philipsen B, Mortier F, et al. A post-mortem survey on end-of-life decisions using a representative sample of death certificates in Flanders, Belgium: Research protocol. BMC Public Health 2008;8:299 [FREE Full text] [CrossRef] [Medline]
  26. Pennec S, Monnier A, Pontone S, Aubry R. End-of-life medical decisions in France: A death certificate follow-up survey 5 years after the 2005 Act of Parliament on patients' rights and end of life. BMC Palliat Care 2012;11:25 [FREE Full text] [CrossRef] [Medline]
  27. Ardilly P. Les Techniques de Sondage [Survey Sampling Techniques]. Paris: Technip; 2006.
  28. Pennec S, Monnier A, Fresson J, Legleye S, Laurent R. Anonymisation et tiers de confiance dans le cadre d’une enquête sur un sujet sensible, les décisions médicales en fin de vie en France [Anonymization and trusted party in a survey on a sensitive topic, medical decisions during end of life in France]. 2012 Mar 12 Presented at: Adelf-Emois; 2012; Dijon (France).
  29. Brouard N. Questionnaire auto-administré via l'Internet avec préservation de l'anonymat absolument nécessaire dans une enquête sur la “fin de vie” [Self-adminstered online questionnaire ensuring anonymity in a survey on end-of-life]. Tanger, Morocco; 2010 Mar 22 Presented at: Sixième Colloque francophone sur les sondages [Sixth french-speaking colloquium on survey sampling]; 2010; Tanger (Morocco).
  30. Institut National D'Études Démographiques. 2015. Enquête “Fin de Vie en France”   URL: [accessed 2015-11-03] [WebCite Cache]
  31. Sautory O. Challenges in Survey Taking for the Next Decade. Statistics Canada. 2003. CALMAR 2: A new version of the CALMAR calibration adjustment program   URL: [accessed 2015-10-29] [WebCite Cache]
  32. American Association for Public Opinion Research (AAPOR). 2011. Standard Definitions: Final Dispositions Of Case Codes And Outcome Rates For Surveys   URL: http:/​/www.​​AAPORKentico/​Education-Resources/​For-Researchers/​Poll-Survey-FAQ/​Response-Rates-An-Overview.​aspx [accessed 2015-10-29] [WebCite Cache]
  33. De Leeuw ED. Choosing the method of data collection. In: De Leeuw ED, Hox JJ, Dillman DA, editors. International Handbook of Survey Methodology. New York: Lawrence Earlbaum Associates; 2008:117-135.
  34. Grava-Gubins I, Scott S. Effects of various methodologic strategies: Survey response rates among Canadian physicians and physicians-in-training. Can Fam Physician 2008 Oct;54(10):1424-1430 [FREE Full text] [Medline]
  35. Scott A, Jeon S, Joyce CM, Humphreys JS, Kalb G, Witt J, et al. A randomised trial and economic evaluation of the effect of response mode on response rate, response bias, and item non-response in a survey of doctors. BMC Med Res Methodol 2011;11:126 [FREE Full text] [CrossRef] [Medline]
  36. Cohen J, Bilsen J, Fischer S, Löfmark R, Norup M, van der Heide A, et al. End-of-life decision-making in Belgium, Denmark, Sweden and Switzerland: Does place of death make a difference? J Epidemiol Community Health 2007 Dec;61(12):1062-1068 [FREE Full text] [CrossRef] [Medline]
  37. Legleye S, Bohet A, Razafindratsima N, Bajos N, Fecond RT, Moreau C. A randomized trial of survey participation in a national random sample of general practitioners and gynecologists in France. Rev Epidemiol Sante Publique 2014 Aug;62(4):249-255. [CrossRef] [Medline]
  38. VanGeest JB, Wynia MK, Cummins DS, Wilson IB. Effects of different monetary incentives on the return rate of a national mail survey of physicians. Med Care 2001 Feb;39(2):197-201. [Medline]
  39. Kellerman SE, Herold J. Physician response to surveys. A review of the literature. Am J Prev Med 2001 Jan;20(1):61-67. [Medline]
  40. VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: A systematic review. Eval Health Prof 2007 Dec;30(4):303-321. [CrossRef] [Medline]
  41. Gautier A. Baromètre Santé Médecins Généralistes [Health Barometer Among General Practitioners]. Saint-Denis (France): INPES; 2011:1-266.

AAPOR: American Association for Public Opinion Research
CCTIRS: Comité Consultatif sur le Traitement de l’Information en matière de Recherche dans le domaine de la Santé
EOLF: End of Life in France
GP: general practitioners
INED: Institut National d’Études Démographiques
INSERM: de l’Institut National de la Santé et de la Recherche Médicale
OR: odds ratio

Edited by G Eysenbach; submitted 18.07.14; peer-reviewed by L Van den Block, E Papavasiliou; comments to author 12.01.15; revised version received 06.03.15; accepted 12.07.15; published 18.02.16


©Stephane Legleye, Sophie Pennec, Alain Monnier, Amandine Stephan, Nicolas Brouard, Johan Bilsen, Joachim Cohen. Originally published in the Interactive Journal of Medical Research (, 18.02.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Interactive Journal of Medical Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.