Public-Private Dialogue

A website of the OPEN PRIVATE SECTOR platform
Created by the World Bank, moderated by CIPE.

Created by the World Bank, moderated by CIPE

  • Facebook
  • Twitter
  • YouTube
  • HOME
  • LESSONS LEARNED
  • COUNTRY CASES
    • Africa
    • Asia
    • Eurasia
    • Latin America & the Caribbean
  • TOOLS
    • PPD Charter
    • PPD Handbook
    • Monitoring and Evaluations
      • The Evaluation Wheel/The Reform Process Table
      • The PPD Reform Tracking Tool
      • Monitoring and Evaluation for Business Environment Reform: A Handbook for Practitioners
      • The PPD Performance Indicators for Donors
    • Operational Resources
      • Manual for Trainers on PPD and Good Governance
      • Quick Guides to Integrating Public-Private Dialogue
      • Key Documents for Setting up PPD Projects
  • WORKSHOP 2015
    • Past Workshops
      • Workshop 2014
      • Workshop 2011
      • Workshop 2010
      • Workshop 2009
      • Workshop 2008
      • Workshop 2007
      • Workshop 2006
  • eLEARNING
  • BLOG
  • ABOUT
    • Contact
You are here: Home / Tools / PPD Handbook / D. Monitoring and Evaluation Framework for PPD / D.3. Methods and Instruments for data collection

D.3. Methods and Instruments for data collection

D.3.1. Inventory and cataloguing of existing documentation

Usually, an inventory is a first step in data collection. It contains a first analysis of existing internal and external information on the PPD. Usually conducted by an external consultant, the inventory will lead to an inception report for the M&E process.

The analysis needs to be focused on the basic questions to be addressed in the M&E exercise defined in step one above. It offers an opportunity to identify and refine sub-questions and corresponding indicators, along with their sources of verification.

The focus in the inventory is on a review of inputs, activities, outputs, outcomes and impacts, based upon the information available in existing documents and archived material. The information can be further analyzed using a small set of key performance criteria:

  • Relevance: addressing the relationship between outputs and outcomes on the one hand, and
    stakeholders’ needs and expectations on the other.
  • Efficiency: addressing the relationship between the inputs and outputs or activities, in
    particular how much input is required for the outputs.
  • Effectiveness: addressing the relationship between the outputs and outcomes – that is, to what degree do the output activities lead to expected outcomes.
  • Sustainability: Addressing the relationship between the outcomes and impact of the PPD and
    the continuation of activities enabled by continued provision of inputs or the identification of new alternative inputs.
  • Quality of the PPD process: Measuring the degree of appreciation of stakeholders of the quality of the partnership and existing measures to ensure quality control and improvement.

If necessary, this list might be extended with other aspects and/or performance criteria.

Here is a checklist for conducting this inventory:

Is there sufficient
information available in
written material? (Yes/No)
Methods and tools
needed to find additional
information
Inputs/activities
Output
Outcome
Impact
Relevance (relationship between
outputs and outcomes and the
needs of stakeholders)
Efficiency (relationship between
inputs and outputs)
Effectiveness (relationship between
output and outcome)
Sustainability (relationship between
outcome and impact and future
inputs/activities)
Quality of the process of the PPD
Other aspects specific to the PPD’s
context

Using this matrix, one can more easily identify where more in-depth research is needed and the tools for undertaking that research. Furthermore, the matrix can be used to make a final check if the M&E process will meet the requirements mentioned in the ToR.

It might, in some cases, lead to changes in the ToR when the matrix shows that certain information requirements cannot possibly be met through the M&E process.

D.3.2. Baseline, benchmark and control-group research

Also taking the form of a desk study, this stage constitutes a more in-depth review of PPD-related documents, including looking at the PPD’s context.

Here is a list of possible relevant documents that might be available:

Internal documents of the
PPD itself
Baseline data on the context
of the PPD
Benchmarking
Original program document PRSP, private sector
development strategies
Research on PPD processes in
other localities
Mission statement, mandate,
official acts
Policy papers of relevant
Ministries
Research on private sector
development in localities
where no PPD-process has
taken place
Rules and regulations Economic surveys M&E reports on PPD processes,
either in the same locality or
elsewhere, where similar
initiatives with similar
objectives and indicators have
taken place
Periodic planning documents Information from bureaus for
statistical research
Control group experiments as
an element in the M&E process
(NB: this will greatly increase
costs)
Periodic reporting documents Enterprise surveys
Minutes of meetings World Bank’s Doing Business
Indicators
Existing internal or external
evaluations
Brochures and publications
Press clippings

It is obvious that the level of effort and costs required to obtain information will increase drastically with each category. In many countries, no reliable baseline studies and statistical data are available.

In most cases it is likely that no systematic benchmark or control-group related information is available. Benchmarking and control group information is likely to require specific instruments to be developed in the M&E process design, and a corresponding budget. Although such experiments are interesting, budget limitations will often preclude them.

D.3.3. PPD surveys

Surveys and questionnaires are an effective way to generate quick information on the opinions of target groups, but usually require a significant amount of time and resources. It is therefore recommended to use standardized surveys, which can be customized by adding or deleting specific questions.

The use of standardized surveys and questions has the additional benefit of enabling comparison of results of PPDs in different contexts.

In conducting and customizing surveys one should take the following aspects into account:

Design Aspect Comments/Suggestions
Open or closed
questions?
  • Closed questions generate quicker information that is easy to process.
  • Providing scales (e.g. from 1 to 5) for answers can enrich information.
  • Open questions are time-consuming to respond to and difficult to
    process.
Amount of time
that can be
requested from
respondents to fill
out questionnaires
  • A good survey should not take more than 30 minutes for people with a
    stake in the evaluation, and 15 minutes for those who are indirectly
    involved.
Language
  • Language should be clear and simple.
  • Think about translations in local languages.
Digital or paper
  • Digital surveys are easy to process and the Internet can be used as a
    tool.
  • In many situations, digital surveys will not be possible because of lack of
    access to technology.
Piloting and
testing
  • Customized surveys should be tested to assess if they can generate
    sufficient information and to make sure questions are understandable
    for respondents. Also, the amount of time needed to fill out surveys
    should be assessed.
Control questions
  • A good survey should contain a some control questions to ensure that
    the information collected is sufficiently reliable
Number of desired
respondents
  • The reliability of results of surveys increases with the number of
    respondents.
  • In the context of PPDs it is important to ensure that desegregated
    information can be generated on each specific stakeholder group.
  • If it is not possible to disseminate surveys widely, they can still be used,
    provided that results can be cross-checked with other methods of data collection.
Timing of
questionnaires
  • One should try to disseminate questionnaires at moments when
    respondents are ready and willing to invest time on them – for example,
    the end of the financial year is generally a poor time to survey
    entrepreneurs.
Level of effort that
is realistic for
analyzing and
reporting data
  • When digital means for processing surveys are available, analyzing and
    reporting on data will usually not require significant time and effort.
  • Open questions can only be processed when significant time is available
    for analysis.
Rolling out the
surveys
  • If surveys are sent to people without proper follow-up, non responsiveness
    can be high. A successful response rate would be at least
    60 percent.
  • Responsiveness can be greatly increased when surveys are collected
    manually or when distributed during events at the end of which the
    surveys can be collected.
  • Download >> PDF (98k)

    Table of Contents

    D.1. Introduction

    D.2. Project management guidelines: three steps toward evaluating PPD

    D.3. Methods and Instruments for data collection

    D.4. Evaluation of organizational process and evolution of the PPD

    D.5. Evaluation of the political economy impact of PPDS on reform processes

    D.6. Evaluation of regulatory and economic outcomes and impact of PPDS

    D.7. Concluding Remarks

    Rights & Permissions

    The material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The sponsors of the work encourage dissemination of its work and will normally grant permission promptly.

    For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the World Bank, DFID or the OECD Development Center.

    ALL RIGHTS RESERVED
    Small and Medium Enterprise Department
    The World Bank Group

    Latest Tweets

    • Loading tweets...

    Follow @PPDialogue

    In Partnership With

    CIPE World Bank Group International Finance Corporation DfID GIZ OECD

    About | Legal | Privacy Policy | Terms Of Use

    All Rights Reserved © 2016

    Copyright © 2023 · Genesis Sample Theme on Genesis Framework · WordPress · Log in