Glossario
Teoria del cambiamento. Approccio controfattuale. Effetti. Distorsioni. Analisi costi-benefici. La valutazione di una politica pubblica, oltre a un suo bagaglio teorico, ha un suo linguaggio specifico. Qui una piccola guida - in costante aggiornamento - ai termini e ai concetti fondamentali.
S
Scrutiny
It is one of the dimensions of parliamentary oversight. In this dimension, the parliament is engaged in a preventative evaluation of governmental conduct that precedes the formal adoption by the government of a decision, action or public policy. Parliaments scrutinise governmental proposals or draft documents to address future policy choices and exercise an influence over the sphere of action of the executive.
For further details, see: Parliamentary oversight
Selection bias
To estimate the effect of a policy, the individuals treated and those not treated should have the same initial characteristics, so that any changes in the result variable can be attributed solely to the public policy. When the initial characteristic are different, however, a selection bias occurs. We speak of bias because it prevents the use, to estimate the effect, of a comparison between the average result variable for those treated and the average for those not treated. The bias is caused by the selection because it depends on differences in the characteristics of treated and the non-treated individuals which existed before the public policy. These influence the allocation of individuals to the treated group or the non-treated group. These influence whether individuals are allocated to the treated group or the non-treated group.
Statistical matching
Statistical matching is a method used to define a control group ex post byselecting, from the individuals excluded from the treatment (non-treated), those most similar (controls) to the individuals admitted to the treatment (treated), on the basis of observable characteristics. If the matching is carried out correctly, the effect of the treatment can be calculated simply as the difference between the average of the result variable for the treated individuals and the matched non-treated individuals. This represents an estimate of the counterfactual.
Statistical matching is said to use observations that have a common support, since it is conducted with individuals from the two most comparable groups on the basis of observable characteristics.
If there are a high number of characteristics to be considered, it can be particularly difficult to find the most similar individuals. To resolve this problem, a propensity score is generally used, i.e. the probability of an individual being treated, given his or her observable characteristics. It has been demonstrated that the essential condition for the elimination of selection bias on the basis of observable characteristics is met even if these characteristics are encapsulated by the propensity score alone.
Studio controllato randomizzato
E' un tipo di valutazione che si basa sul metodo sperimentale. Fin dal 1944, anno di pubblicazione dei risultati dell'esperimento sulla patulina del British Medical Research Council, costituisce il fondamento della ricerca clinica in medicina. In casi circoscritti e con numerosi limiti, lo studio controllato randomizzato (Randomized Controlled Trial, RCT) può essere applicato anche alla valutazione delle politiche pubbliche.
Il metodo consiste nella suddivisione di un gruppo di individui in due sottogruppi mediante assegnazione rigorosamente casuale (random) dei componenti. I membri del primo gruppo, detto "sperimentale", ricevono il "trattamento" (ad esempio, l'erogazione di un servizio o la concessione di un beneficio), mentre sono esclusi dalla politica i membri del secondo gruppo, che per questo viene detto "di controllo". Se il campione di partenza è sufficientemente grande, i due gruppi creati risultano statisticamente equivalenti sia nelle caratteristiche osservabili che nelle caratteristiche non osservabili. Ciò implica che i due sottogruppi siano privi di differenze di partenza e che, quindi, la distorsione da selezione sia stata eliminata. Il gruppo di controllo così creato rappresenta la ricostruzione più affidabile della situazione controfattuale.
Ciò consente di stimare l'effetto del trattamento semplicemente confrontando il valore medio della variabile risultato nei due sottogruppi.
Le desiderabili caratteristiche statistiche rendono tale metodo un punto di riferimento ideale nella valutazione controfattuale delle politiche pubbliche.
Synthetic control
The evaluation of the effects of a treatment (such as, for example, a public policy) requires the comparison of the treated subjects to that of other control subjects, in order to measure the difference that the treatment achieves in the value of the specific outcome variable.
The choice of control subjects is a crucial aspect of this process. Ideally, treated and untreated subjects should have the same characteristics, otherwise the difference in the outcome variable could simply reflect the structural differences between the subjects rather than the effect of the treatment.
The "synthetic control" method involves the construction of "synthetic" comparison group, whose characteristics are obtained as a weighted average, in which the weighting is designed in a manner to faithfully reflect the characteristics of the entire population of subjects prior to treatment. The synthetic control thus constructed performs as a credible counterfactual of the true scenario.
The method combines elements from the statistical techniques of "matching" and "difference in differences". Compared to the latter, the synthetic control has the advantage of making the choice of weights explicit and transparent.
Because it allows a systematic approach to the selection of comparative subjects, the synthetic control method has been used effectively in political science, health policy evaluation, criminology and economics.