Methodology of the Knowledge Networks Survey of Public and Private Schooling Graduates
The Cardus Education Study includes a survey of randomly selected Americans that was administered by Knowledge Networks. This survey included a large oversample of private school graduates that was selected from the Knowledge Networks internet panel. The random sample was limited to respondents between 23 and 40 years of age who had graduated from high school.
Schooling history information included whether the respondent had primarily attended a Catholic, conservative Protestant or "Christian school," another type of Christian school, a non-Christian religious school, a nonreligious school, or a home school. About 1,000 of the respondents had primarily attended a private elementary or high school, and about 500 had primarily attended public school.
The question used to classify respondents was, "Which type of school did you primarily attend for high school?" Respondents were given the following categories to choose from: public, Catholic, religious school (NOT Catholic), non-religious private, and home school. Those who reported a non-Catholic religious school received a follow-up question that asked, "Which type of religious school did you primarily attend for high school?" Response categories were Conservative Protestant or "Christian school", Other Christian, Jewish, or Other.
Respondents completed a half-hour survey that included questions on schooling history and experiences, evaluation of their high school, family background, occupational goals, current education and occupation, marriage and family, religious and spiritual involvement, and civic and political engagement. KN profile and public affairs information previously collected about panel respondents was also merged into the survey dataset. In addition, since the survey asked the college or university attended by the respondent, government data on colleges and universities available in the Integrated Postsecondary Education Survey was merged into the final dataset.
The resulting datasets were analyzed using nested regression analyses to predict outcomes related to socio-economic status, personality and mental health, prosocial attitudes and behavior, including volunteer, civic engagement, and political participation. For each dependent variable, the first model included binary variables related to type of schooling—private elementary and public high school, Catholic high school, conservative Protestant or "Christian school," other Christian school, nonreligious school, and home school. Those who primarily attended public elementary and high school served as the comparison group. The home school category is split into home schoolers whose mother attended religious services regularly and those whose mother did not attend religious services regularly. A binary variable is then entered into the models for "religious" homeschoolers and "nonreligious" homeschoolers.
The distribution of private school respondents in the preliminary analysis is as follows:873 Public
283 Catholic
124 Conservative Protestant or "Christian School"
109 Nonreligious
61 Religious home school
21 Nonreligious home school
(Note that this distribution may change slightly in later analyses since we directly asked each respondent what high school they graduated from and can use that information to confirm the respondent's school type.)
The size of each of the private schooling groups is modest, but adequate for most of our purposes. Note that the large private schooling oversample is highly unusual in comparison with other surveys. In a random survey of 2,000 Americans (without oversampling), we would expect to have only 17 rather than 124 respondents who report that they graduated from high school, are between 23 and 40, and primarily attended a conservative Protestant high school.
When evaluating the adequacy of sample sizes, it is also important to keep in mind that tests of statistical significance take into account the sample size. A small sample size makes it more difficult to uncover statistically significant differences between two groups (or subpopulations), but it doesn't lead to biased results per se. In a random sample, finding a statistically significant difference between, say, average years of education of public schoolers and Christian schoolers depends on 1) the sample size of the two groups, 2) variation in years of education within each group, and 3) the estimated size of the difference in years of education between the two groups. In some of our comparisons, the size of the difference between Christian schoolers and public schoolers is large enough to overwhelm the other two factors (namely, sample size and within-group variation). In other cases, the within-group variation is relatively small (i.e., nearly all Christian schoolers are similar on particular variables), which also contributes to finding statistically significant differences between Christian schoolers and public schoolers despite small sample sizes. In these cases, obtaining a larger random sample of Christian schoolers is very unlikely to change our conclusions about differences between public schoolers and Christian schoolers.
The second model added demographic variables, including gender, race, and age, as well as numerous family background characteristics. Specifically, the education, religious tradition, religious service attendance, and volunteering habits of respondent's mother and father are included as controls. How close the respondent was to their mother and father and whether their father or mother pushed the respondent academically is also included in the models. The marital status and living arrangement when the respondent was in high school is also controlled. In particular, a variable is included for respondents who were raised by both biological parents and variables for the number of years that respondents lived with each biological parent.
In this preliminary analysis, the models did not include variables for current characteristics of respondents, such as their current religious service attendance, marital status, income, or education. These variables are not included since most of them are used as outcome or dependent variables. And, in most other cases, these variables likely mediate the relationship between schooling background and the outcome of interest. For example, one's current education likely influences participation in the political process. The more educated are more likely to be involved in politics. One's schooling history likely influences educational attainment, which in turn influences political involvement. Thus far, however, our analysis has only focused on the direct effects of schooling history on political participation, rather than accounting for the indirect effect that runs through educational attainment.
Missing values were imputed using multiple imputation techniques. Appropriate regression models (linear, logistic, and multinomial) were used depending on the type of dependent variable. The first regression model for each dependent variable, which included only the binary variables for high school type, included weights appropriate for comparing school sectors. Coefficients from the regression models are presented in the graphs in this report. Smaller coefficients or those with higher standard errors should be considered essentially identical to zero (this is reflected in the discussion of findings in this report). Given the small sample sizes of all but the Catholic sector, coefficients in the tables are considered significant if the p-value is less than .1.
Supporting Information
Questionnaire:
Which type of school did you primarily attend for . . .
Which type of religious school did you primarily attend for . . .
| California Table I | Dec. 2007 | A group of 37 Christian educational and cultural leaders representing some of the diversity of the sector meets in California for a symposium that explores some of the questions at the interface between Christian education and culture. The symposium leads to the publishing of a discussion paper and ultimately to this study. |
| Funding Proposals | June-Oct. 2008 | A formal funding proposal is developed and presented to various potential funders. An initial $850,000 is secured from the RDV Foundation, William Voortman Foundation, and Van Lunen Foundation. An Advisory Committee of leading experts is recruited to assist in the shaping of the project. |
| Advisory Group Convenes | Mar. 2009 | The Advisory Group meets to connect this project with other research on the subject in North America. |
| Quantitative Study Commissioned | May 2009 | Cardus contracts the University of Notre Dame, led by Dr. David Sikkink, to study the largest ever representative sample of Christian school graduates in North America. Web-based surveys are developed and promoted within the broader Christian education community with the aim of securing a large data sample. The surveys utilize questions from previous surveys to assist in comparative analysis. The same surveys are distributed using representative sampling methods so that there are controls in place to assist in the analysis and interpretation of the data. |
| Qualitative Studies Commissioned | Oct. 2009 | In partnership with Redeemer University College, Dr. Deani Neven Van Pelt leads a process commissioning a series of smaller research studies are ccommissioned through an award process. These studies are intended to add depth to the questions asked on the quantitative survey and explore issues which do not lend themselves to quantitative study. |
| California Table II | Dec. 2009 | The second roundtable event takes place in Huntington Beach, California. As the data begin to come into focus, the event's agenda seeks early feedback from education stakeholders on what the findings will mean for Christian education. |
| CES Washington Media Launch | May 2011 | Cardus with the University of Notre Dame hosts a lunch for think tank and media leaders at the Willard Hotel in D.C., billed as "the release of the most substantive study ever of Christian schools in North America." |
| CES Washington Results Pre-Launch | May 2011 | Twenty-five educational leaders engage the pre-release findings in peer discussion. |
| CES Phase I Released | Aug. 2011 | The Phase I report, with two sets of curriculum, are released for free public distribution. |
The Advisory Group was formed to connect this project to other research on the subject in North America. The Advisory Group met in Chicago on March 2009, and again in December 2010, receiving the various research reports, identifying common themes, and guiding the conclusions and recommendations that flow from the data. The Advisory Group consists of:
Augustine. (1958). On Christian doctrine. Prentice-Hall.
Barna, G. (2001). Real teens: A contemporary snapshot of youth culture. Regal.
Baumeister, R. (January 2005). Exploding the self-esteem myth. Scientific American.
Basham, P. (2007). Homeschooling: From the extreme to the Mainstream, 2nd Edition. The Fraser Institute.
Bauer, S. (2003). The well-educated mind: A guide to the classical education you never had. Norton.
Bauer, S. and Wise, J. (2004). The well-trained mind: A guide to classical education at home. Norton.
Bennett, N. (1976). Teaching styles and pupil progress. Harvard.
Berkhof, L. & Van Til, C. (1990). Foundations of Christian education: Addresses to Christian teachers. P&R.
Berquist, L. (1998). Design your own classical curriculum: A Guide to Catholic Home Education. Ignatius.
____. (1999). The harp and laurel wreath: Poetry and Dictation of the Classical Curriculum. Ignatius.
Berry, W. (1987). Home economics. North Point.
____. (1990). What are people for? North Point.
Bradshaw, T. and Nichols, B. (2004). Reading at risk: A survey of literary reading in America. National Endowment for the Arts, Washington, DC.
Brooks, D. (April 2001). The organization kid. Atlantic Monthly.
Brophy, J. and Good, T.L. (1986). "Teacher behavior and student achievement." Handbook of Research and Teaching. Macmillan: 328-375.
Braum, H., Jenkins, F., and Griggs, W. (July, 2006). "National assessment of educational progress: Comparing private schools and public schools using hierarchical linear modeling." U.S. Department of Education.
Bunting, J. (1998). An education for our time. Regnery.
Canada, G. (1998). Reaching up for manhood: Transforming the lives of boys in America. Beacon.
Chall, J. (2000). The academic achievement challenge: What really works in the classroom. Guilford.
Clark, C. (2004). Hurt: Inside the world of today's teenagers (Youth, Family, and Culture). Baker.
Clark, C. (2007). Disconnected: Parenting teens in a MySpace world. Baker.
Collins, M. (1982). The Marva Collins way. Penguin Putnam.
Cowan, L. and Guinness. G. (1998). Invitation to the classics: A guide to books you've always wanted to read. Baker.
Dabney, R. (1996). On secular education. Canon.
Davis, S., Jenkins, G., and Hunt, R. (2002). The Pact: Three young men make a promise and fulfill a dream. Riverhead.
Dawson, C. (1989). The crisis of western education. Franciscan.
Drexler, J. L. (Ed.) (2007). Schools as communities: Educational leadership, relationships, and the eternal value of Christian schooling. Purposeful Design.
Edmondson, H. (2005). John Dewey and the decline of American education: How the patron saint of schools has corrupted teaching and learning. ISI.
Edmundson, M. (2004). Why read? Bloomsbury.
Egan, K. (2002). Getting it wrong from the beginning: Our progressivist inheritance from Herbert Spencer, John Dewey, and Jean Piaget. Yale.
Esquith, R. (2003). There are no shortcuts. Pantheon.
Flesch, R. (1986). Why Johnny can't read: And what you can do about it. Harper.
Friedman, T. (2007). The world is flat: A brief history of the twenty-first century. Picador.
Fullan, M. (2007). Leading in a culture of change. Jossey-Bass.
Gaebelein, F. (1954). The pattern of God's truth: Problems of integration in Christian education. Oxford.
____. (1985). The Christian, the arts, and truth: Regaining the vision of greatness. Multnomah.
____. (1995). Christian education in a democracy. Purposeful Design.
Gamble, R. (Ed.) (2007). The great tradition: Classic readings on what it means to be an educated human being. ISI.
Garber, S. (1996). The fabric of faithfulness: Weaving together belief and behavior during the university years. InterVarsity.
Gay, C. (1998). The way of the (modern) world: or, why it's tempting to live as if God doesn't exist. Eerdmans.
Glenn, Charles (2011).Contrasting Models of State and School: A Comparative Historical Study of Parental Choice and State Control. Continuum.
Glenn, Charles (2011). American Indian/First Nations Schooling: From the Colonial Period to the Present. Palgrave Macmillan.
Glenn, Charles (2011). African-American/Afro-Canadian Schooling: From the Colonial Period to the Present. Palgrave Macmillan.
Giussani, L. (1995). The risk of education: Discovering our ultimate destiny. Crossroad.
Grant, G. (1969). Technology and empire: Perspectives on North America. Anansi.
Gregory, M. (2007). The seven laws of teaching. FQ.
Guinness, O. (1994). Fit bodies, fat minds: Why evangelicals don't think and what to do about it. Baker.
____. (1998). The call: Finding and fulfilling the central purpose of your life. Word.
Guinness, O. and Seel, J. (1992). No God but God: Breaking with the idols of our age. Moody.
Gunton, C. (1993). The one, the three and the many: God, creation and the culture of modernity. Cambridge.
Guroian, V. (1998). Tending the heart of virtue: How classic stories awaken a child's moral imagination. Oxford.
Gurian, M. (1996). The wonder of boys: What parents, mentors, and educators can do to raise boys into exceptional young men. Putnam.
____. (1999). A fine young man: What parents, mentors and educators can do to shape adolescent boys into exceptional men. Putnam.
Gurian, M. and Stevens, K. (2005). The minds of boys: Saving our sons from falling behind in school and life. Jossey Bass.
Hartley-Brewer, E. (2001). Raising confident boys: 100 tips for parents and teachers. DaCapo.
Hauerwas, S. (1981). A community of character. Notre Dame.
Heath, B. (2006). Millstones & stumbling blocks: Understanding education in post-Christian America. Fenestra.
Heck, J. (2005). Irrigating deserts: C.S. Lewis on education. Concordia.
Henry, D. and Beaty, M. (Ed.). (2006). Christianity and the soul of the university: Faith as a foundation for intellectual community. Baker.
Henry, W. (1994). In defense of elitism. Doubleday.
Hirsch, E. (1988). Cultural literacy. Houghton Mifflin.
____. (1996). The Schools We Need: And Why We Don't Have Them. New York: Doubleday.
____. (2006). The Knowledge Deficit. Hougton Mifflin.
Hoff-Sommers, C. (2000). The war against boys: How misguided feminism is harming our young men. Simon & Schuster.
Holmes, A. (2001). Building the Christian academy. Eerdmans.
Hunter, J. (2000). The death of character: Moral education in an age without good or evil. Basic.
Isaacson, W. (2007). Einstein: His life and universe. Simon & Schuster.
Jeffrey, D. (October 2007). The pearl of great wisdom: The deep & abiding biblical roots of western liberal education. Touchstone.
Keller, T. (2008). The reason for God: Belief in an age of skepticism.
Kilpatrick, W. (1993). Why Johnny can't tell right from wrong: And what we can do about it.
Kindlon, D. and Thompson, M. (2000). Raising Cain: Protecting the emotional lives of boys. Ballantine.
Kinnaman, D. (2007). UnChristian: What a new generation really thinks about Christianity and why it matters. Baker.
Kuyper, A. (1943). Lectures on Calvinism. Eerdmans.
Lakoff, G. (2004). Don't think of an elephant: Know your values and frame the debate - The essential guide for progressives. Chelsea Green.
Lakoff, G. and Johnson, M. (1999). Philosophy in the flesh : The embodied mind and its challenge to western thought. Basic.
Lareau, A. (2003). Unequal childhoods: Class, race, and family life. California.
Lewis, C. (1955). The abolition of man. Macmillan.
____. 1962. "Bluspels and Flalansferes," The Importance of Language, Max Black (Ed.). Prentice-Hall.
____. (1962). "De descriptione temporum." They asked for papers: Papers and addresses. Geoffrey Bles.
____. (1967). Christian reflections. Eerdmans.
Lewis, R. (1997). Raising a modern day knight: A father's role in guiding his son to authentic manhood. Tyndale.
Littlejohn, R. and Evans, C. (2006). Wisdom and eloquence: A Christian paradigm for classical learning. Crossway.
Lockerbie, D. (1972). The way they should go. Oxford.
____. (1989). Thinking-and-acting like a Christian: Love the Lord your God with all your mind. Multnomah.
____. (2005). A Christian paideia: The habitual vision of greatness. Purposeful Design.
Lockerbie, D. (2007). A passion for learning: The history of Christian thought on education. Purposeful Design.
Luce, R. (2005). Battle cry for a generation: The fight to save America's youth. Cook.
Malloch, T. and Massey, S. (2006). Renewing American culture: The pursuit of happiness. Scrivener.
Machen, J. (2004). Education, Christianity and the state. Trinity.
MacIntyre, A. (1984). After virtue. Notre Dame.
Machen, J. (1969). Christianity and culture. Banner of Truth.
Malik, C. (1980). The two tasks. Crossway.
Marrou, H.I. (1956). The history of education in antiquity. Sheed & Ward.
Marshall, P. (1998). Heaven is not my home: Living in the now of God's creation. Word.
McClay, W. (1994). The masterless: Self and society in modern America. North Carolina.
Milton, J. (1975). Milton on Education. Ams.
Moreland, J. (1997). Love your God with all your mind. Navpress.
Myers, K. (1989). All God's children and blue suede shoes: Christians and popular culture. Crossway.
Naugle, D. (2002). Worldview: The history of a concept. Eerdmans.
Newberger, E. (1999). The men they will become: The nature and nurture of male character. Da Capo.
Newkirk, T. (2002). Misreading masculinity: Boys, literature, and popular culture. Heinemann.
Newman, J. (1999). The idea of a university. Regnery.
Nisbet, R. (1971). The degradation of the academic dogma. Basic.
Noebel, D.A. (2006). Understanding The times (revised 2nd edition). Summit.
Noll, M. (1994). The scandal of the evangelical mind. Eerdmans.
Palmer, P. (1993). To know as we are known: Education as a spiritual journey. HarperSanFrancisco.
____. (1998). The courage to teach: Exploring the inner landscape of a teacher's life. Jossey-Bass.
____. (1999). Let your life speak: Listening for the voice of vocation. Jossey-Bass.
Payne, L. (1979). Real presence: The Christian worldview of C.S. Lewis as incarnational reality. Crossway.
Pearcy, L. (2005). The grammar of our civility: Classical education in America. Baylor.
Pennings, R. (2005). "What is to be done . . . in the public square?". Comment.
Pennings, R. (2004). "Work and Vocation" in Cornelis Van Dam (ed.) Work and Leisure in the Life of a Christian. Burlington Reformed Study Center.
Perkins, J. (1976). Let justice roll down. Regal.
____. (1982). With justice For all: A strategy for community development. Regal.
____. (1993). Beyond charity: The call to Christian community development. Baker.
Perkins, J. with Kadlecek, J. (1995). Resurrecting hope. Regal.
Perkins, J. (Ed.) (1995). Restoring at-risk communities: Doing it together and doing it right. Baker.
Polanyi, M. (1958). Personal knowledge: Towards a post-critical philosophy. Chicago.
Pollack, W. and Pipher, M. (1999). Real Boys: Rescuing Our Sons from the Myths of Boyhood. Owl.
Postman, N. (1992). Technopoly: The surrender of culture to technology. Vintage.
____. (1995). The end of education. Vintage.
Ravitch, D. (2000). Left back: A century of failed school reforms. Simon & Schuster.
Riesen, R. (2002). Piety and philosophy: A primer for Christian schools. ACW.
Rosen, C. (2005). My fundamentalist education: A memoir of a divine girlhood. PublicAffairs.
Salomone, R. (2003). Same, different, equal: Rethinking single-sex schooling. Yale.
Sax, L. (2005). Why gender matters: What parents and teachers need to know about the emerging science of sex differences. Doubleday.
Sayers, D. (1948). The lost tools of learning. Methuen.
Schall. J. (1988). Another sort of learning. Ignatius.
Schmoker, M. (1999). Results: The key to continuous school improvement. Association for Supervision & Curriculum Development.
(2004). Black boys: The litmus test for public school education. Schott Foundation for Public Education
Schultze, Q. (2005). Here I am: Now what on earth should I be doing? Baker.
Seerveld, C. (1980). Rainbows for a fallen world. Shiloh.
Simmons, T. (2002). Climbing parnassus: A new apologia for Greek and Latin. ISI.
Sire, J. (2004). Naming the elephant: Worldview as a concept. InterVarsity.
Smith, C. (2005). Soul searching: The religious and spiritual lives of American teenagers. Oxford.
Stob, L. (2007). Growing Christian leaders: A vision and call for purposeful Christian education.
Stob, L. (2007). The model Christian school: Governing for the ideal.
Suskind, R. (1998). A hope in the unseen. Broadway.
Taylor, C. (1989). Sources of the self: The making of the modern identity. Harvard.
Taylor, J. 1998. Poetic knowledge: The recovery of education. State University of New York.
Thompson, M. (2000). Speaking of boys: Answers to the most-asked questions about raising sons. Ballantine.
Van Pelt, D. 2004. Home education in Canada: A report on the pan-Canadian study on home education 2003. Canadian Centre for Home Education.
Van Pelt, D., Allison, P., and Allison D. (May 2007). Ontario's private schools: Who chooses them and why? The Fraser Institute.
Van Pelt, M., Pennings, R., and Van Pelt, D. (November 2007). Faithful and fruitless in Ontario: Status quo in education policy. Policy Options: 25-26.
Walsh, B. and Middleton, R. (1984). The transforming vision: Shaping a Christian worldview. InterVarsity.
Weaver, R. (1984). Ideas have consequences. Chicago.
Weiner, D. and Keylor, W. (1976). From Parnassus: Essays in honor of Jacques Barzun. Harper & Row.
Wilson, D. (1991). Recovering the lost tools of learning: An approach to distinctively Christian education. Canon.
____. (1996). Repairing the ruins. Canon.
____. (1999). The paideia of God. Canon.
____. (2001). Excused absence: Should Christian kids leave public schools? Canon.
____. (2002). The case for classical Christian education. Crossway.
Wittmer, M. (2004). Heaven is a place on earth: Why everything you do matters to God. Zondervan.
Wolterstorff, N. (1980). Educating for responsible action. Eerdmans.
Wright, N. (1997). For all God's worth: True worship and the calling of the church. Eerdmans.
Zemelman, S., Daniels, H., and Hyde, A. (1993). Best practice. Heineman.
Zengotita, T. (2005). Mediated: How the media shapes your world and the way you live in it. Bloomsbury.
Zoch, P. (2004). Doomed to fail: The built-in defects of American education. Ivan R. Dee.
Register for the CES e-newsletter (free).
Do Christian schools deliver on their promises? Cardus reports on the largest-ever sample of Christian school graduates and administrators in North America, focusing on students' spiritual formation, cultural engagement, and academic development. Available free of charge in digital form, and at cost in print form.
Click here to download CES docs immediately, or to purchase printed book(s) at cost.
We'd love to hear from you! After you've finished the four-part CES curriculum (either set), click here to report back on your school's discussions. Is there a gap between your school's vision and its current reality? Is your school aiming to produce world changers? Is it doing so? Did your group recommend changes to your school's program?
When a discomfiting question suddenly accosts you for spare change, I believe that a truly free academic gives it welcome as a stranger.