About
OSF example survey
Uploaded 1 mo ago
by RobinHorton
- Object Notebook
Download
Recreating surveys in EDSL¶
This notebook provides code for recreating and generating responses to a survey using EDSL, an open-source library for simulating surveys, experiments and other research using AI agents and large language models. Please see our documentation page for instructions and tutorials on using the library.
Example survey used in this notebook: QRP Training for Grad Students: Ethics_Training_Time_1_Version_1 (Center for Open Science, published by OSF).
Link: https://osf.io/z9tnc
In [1]:
from edsl import Agent, QuestionMultipleChoice, QuestionFreeText, QuestionLinearScale, Survey
Design an Agent to answer the survey¶
Create an Agent or AgentList to answer a survey by passing a dictionary of any desired or relevant traits; you can optionally include a special instruction. Learn more about designing AI agents.
In [2]:
agent = Agent(
traits = {
"persona":"You are a graduate student in the social sciences.",
"age":28,
"gender":"Male",
"location":"Massachusetts"
},
instruction = "Answer each question honestly with respect to your own personal views."
)
Format each Question in EDSL¶
See examples of all available question types.
In [3]:
q_QID44 = QuestionMultipleChoice(
question_name = "QID44",
question_text = "Acknowledging another's technical assistance in publication without that person's permission.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID76 = QuestionMultipleChoice(
# question_name = "QID76",
# question_text = "Acknowledging another's technical assistance in publication without that person's permission.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID32 = QuestionMultipleChoice(
question_name = "QID32",
question_text = "Adding additional research participants because the results collected thus far are not yet statistically significant.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID64 = QuestionMultipleChoice(
# question_name = "QID64",
# question_text = "Adding additional research participants because the results collected thus far are not yet statistically significant.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID56 = QuestionMultipleChoice(
question_name = "QID56",
question_text = "Adding or deleting co-authors to increase chances of acceptance.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID101 = QuestionFreeText(
question_name = "QID101",
question_text = "Before we get started, please answer some questions about yourself."
)
q_QID43 = QuestionMultipleChoice(
question_name = "QID43",
question_text = "Changing the design, methodology or results of a study to please a sponsor.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID75 = QuestionMultipleChoice(
# question_name = "QID75",
# question_text = "Changing the design, methodology or results of a study to please a sponsor.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID48 = QuestionMultipleChoice(
question_name = "QID48",
question_text = "Concealing data or results that contradict one’s own previous research.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID35 = QuestionMultipleChoice(
question_name = "QID35",
question_text = "Deciding whether to include or exclude data after looking at the impact of doing so on the results.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID67 = QuestionMultipleChoice(
# question_name = "QID67",
# question_text = "Deciding whether to include or exclude data after looking at the impact of doing so on the results.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID40 = QuestionMultipleChoice(
question_name = "QID40",
question_text = "Deliberately delaying reporting results in order to publish findings in a higher impact journal.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID72 = QuestionMultipleChoice(
# question_name = "QID72",
# question_text = "Deliberately delaying reporting results in order to publish findings in a higher impact journal.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID38 = QuestionMultipleChoice(
question_name = "QID38",
question_text = "Drawing strong inferences from statistically significant but underpowered results.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID70 = QuestionMultipleChoice(
# question_name = "QID70",
# question_text = "Drawing strong inferences from statistically significant but underpowered results.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID55 = QuestionMultipleChoice(
question_name = "QID55",
question_text = "Failing to acknowledge significant sources of financial assistance.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID47 = QuestionMultipleChoice(
question_name = "QID47",
question_text = "Failing to attain appropriate informed consent.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID52 = QuestionMultipleChoice(
question_name = "QID52",
question_text = "Failing to disclose all potentially relevant conflicts of interest.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID36 = QuestionMultipleChoice(
question_name = "QID36",
question_text = "Failing to report all of a study’s outcome measures.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID68 = QuestionMultipleChoice(
# question_name = "QID68",
# question_text = "Failing to report all of a study’s outcome measures.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID21 = QuestionLinearScale(
question_name = "QID21",
question_text = "For a researcher, how important is choosing a sample size before running a study?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID23 = QuestionLinearScale(
question_name = "QID23",
question_text = "How important are decisions in data collection, analysis, and reporting in affecting how likely a research will find a significant effect?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID27 = QuestionLinearScale(
question_name = "QID27",
question_text = "How important are research ethics to you?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID28 = QuestionLinearScale(
question_name = "QID28",
question_text = "How important is it for you to avoid questionable research practices?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID25 = QuestionLinearScale(
question_name = "QID25",
question_text = "How important is it that results from a psychology study are counterintuitive (e.g., different from what you would expect)?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID29 = QuestionLinearScale(
question_name = "QID29",
question_text = "How important is it to consider the impact of questionable research practices on science?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID22 = QuestionLinearScale(
question_name = "QID22",
question_text = "How important is it to make data publicly available so that results can be verified by other researchers?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID24 = QuestionLinearScale(
question_name = "QID24",
question_text = "How important is it to report studies that 'don't work out'?",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Not At All Important", 7:"Very Important"}
)
q_QID81 = QuestionMultipleChoice(
question_name = "QID81",
question_text = "How would you describe yourself? (Choose one or more of the following)",
question_options = [
"Asian or Asian American",
"Caucasian",
"African American",
"Hispanic or Latino",
"Other"
]
)
q_QID90 = QuestionLinearScale(
question_name = "QID90",
question_text = "I am completely rational.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID91 = QuestionLinearScale(
question_name = "QID91",
question_text = "I am confident in my judgments.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID85 = QuestionLinearScale(
question_name = "QID85",
question_text = "I am not always honest.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID98 = QuestionLinearScale(
question_name = "QID98",
question_text = "I avoid listening.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID89 = QuestionLinearScale(
question_name = "QID89",
question_text = "I can't make up my mind.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID100 = QuestionLinearScale(
question_name = "QID100",
question_text = "I don't gossip.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID92 = QuestionLinearScale(
question_name = "QID92",
question_text = "I have doubted my ability as a lover.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID97 = QuestionLinearScale(
question_name = "QID97",
question_text = "I have said something bad about a friend.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID95 = QuestionLinearScale(
question_name = "QID95",
question_text = "I have taken advantage of someone.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID86 = QuestionLinearScale(
question_name = "QID86",
question_text = "I know why I like things.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID8 = QuestionLinearScale(
question_name = "QID8",
question_text = "I like psychology.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID94 = QuestionLinearScale(
question_name = "QID94",
question_text = "I never cover up mistakes.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID88 = QuestionLinearScale(
question_name = "QID88",
question_text = "I never regret my decisions.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID99 = QuestionLinearScale(
question_name = "QID99",
question_text = "I never take things.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID93 = QuestionLinearScale(
question_name = "QID93",
question_text = "I sometimes tell lies.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID96 = QuestionLinearScale(
question_name = "QID96",
question_text = "I sometimes try to get even.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID9 = QuestionLinearScale(
question_name = "QID9",
question_text = "I think psychological research is similar to research in fields like chemistry, physics, or biology.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID10 = QuestionLinearScale(
question_name = "QID10",
question_text = "I think psychological research is similar to research in fields like philosophy, literature, or modern languages.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID11 = QuestionLinearScale(
question_name = "QID11",
question_text = "I think psychology is a 'soft' science.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID12 = QuestionLinearScale(
question_name = "QID12",
question_text = "I trust the results of studies done by psychologists.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID83 = QuestionMultipleChoice(
question_name = "QID83",
question_text = "In what doctoral program are you?",
question_options = [
"Brain and Behavior",
"Clinical",
"Counseling",
"School"
]
)
q_QID84 = QuestionMultipleChoice(
question_name = "QID84",
question_text = "In what year are you currently in your program?",
question_options = [
"First Year",
"Second Year",
"Third Year",
"Fourth Year",
"Fifth Year",
"Sixth Year"
]
)
q_QID19 = QuestionLinearScale(
question_name = "QID19",
question_text = "It is important for a researcher to report all measures and experimental conditions that were included in a study.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID87 = QuestionLinearScale(
question_name = "QID87",
question_text = "It's hard to shut off a disturbing thought.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID59 = QuestionMultipleChoice(
question_name = "QID59",
question_text = "Overlooking or ignoring others’ questionable research practices.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID57 = QuestionMultipleChoice(
question_name = "QID57",
question_text = "Overlooking or ignoring others’ research misconduct.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID4 = QuestionFreeText(
question_name = "QID4",
question_text = "Please create for yourself a unique identifier that will allow us to link your data throughout the duration of this study; you will be assessed on these data three times total."
)
q_QID60 = QuestionMultipleChoice(
question_name = "QID60",
question_text = "Providing a biased peer review of a manuscript submitted to a journal for publication in order to delay its publication so as to benefit one’s own research.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID42 = QuestionMultipleChoice(
question_name = "QID42",
question_text = "Publishing results of a single study as several articles simply to increase the number of publications derived from the research (the so-called 'salami slicing' problem).",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID74 = QuestionMultipleChoice(
question_name = "QID74",
question_text = "Publishing results of a single study as several articles simply to increase the number of publications derived from the research (the so-called 'salami slicing' problem).",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID20 = QuestionLinearScale(
question_name = "QID20",
question_text = "Questionable research practices have a considerable negative impact on science.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID51 = QuestionMultipleChoice(
question_name = "QID51",
question_text = "Re-use of one’s own previously published data, results, or analysis without citation.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID41 = QuestionMultipleChoice(
question_name = "QID41",
question_text = "Re-use of one’s own previously published ideas or words without citation, such as parts of a literature review section, introduction or methodology, but without re-using data, results, or analysis.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID73 = QuestionMultipleChoice(
question_name = "QID73",
question_text = "Re-use of one’s own previously published ideas or words without citation, such as parts of a literature review section, introduction or methodology, but without re-using data, results, or analysis.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID61 = QuestionMultipleChoice(
question_name = "QID61",
question_text = "Refusing to provide experimental data to a co-author.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID58 = QuestionMultipleChoice(
question_name = "QID58",
question_text = "Refusing to share data or materials with other researchers to prevent questions about the quality of your work from being raised.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID14 = QuestionLinearScale(
question_name = "QID14",
question_text = "Replication of research is only a problem in the field of psychology.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID45 = QuestionMultipleChoice(
question_name = "QID45",
question_text = "Reporting an unexpected result as having been hypothesized from the start.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID77 = QuestionMultipleChoice(
question_name = "QID77",
question_text = "Reporting an unexpected result as having been hypothesized from the start.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID17 = QuestionLinearScale(
question_name = "QID17",
question_text = "Researchers who perform replication studies are not qualified to conduct psychological research.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID34 = QuestionMultipleChoice(
question_name = "QID34",
question_text = "Rounding off a p value simply to make results seem more significant, such as by reporting a p value of .044 to be p = .04.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID66 = QuestionMultipleChoice(
question_name = "QID66",
question_text = "Rounding off a p value simply to make results seem more significant, such as by reporting a p value of .044 to be p = .04.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID39 = QuestionMultipleChoice(
question_name = "QID39",
question_text = "Selective reporting of subgroups, outcomes, and time points.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID71 = QuestionMultipleChoice(
# question_name = "QID71",
# question_text = "Selective reporting of subgroups, outcomes, and time points.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID37 = QuestionMultipleChoice(
question_name = "QID37",
question_text = "Selectively discussing only studies that supported the hypothesized result(s).",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID69 = QuestionMultipleChoice(
# question_name = "QID69",
# question_text = "Selectively discussing only studies that supported the hypothesized result(s).",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID33 = QuestionMultipleChoice(
question_name = "QID33",
question_text = "Stopping collecting data earlier than planned because the hypothesized result already had been attained.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID65 = QuestionMultipleChoice(
# question_name = "QID65",
# question_text = "Stopping collecting data earlier than planned because the hypothesized result already had been attained.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID18 = QuestionLinearScale(
question_name = "QID18",
question_text = "Studies that receive a lot of media attention are often reliable.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID5 = QuestionFreeText(
question_name = "QID5",
question_text = "Study Links Warm Hands, Warm Heart. If you're going out on a date anytime soon, you may find this bit of science news useful: In a carefully controlled experiment, people who held something warm were more likely to perceive someone else as emotionally \"warm\" – and they were more likely to behave in a friendly, generous way themselves. So order the soup, not the salad."
)
q_QID54 = QuestionMultipleChoice(
question_name = "QID54",
question_text = "Submitting a manuscript in which the sponsor was involved in data analysis and/or manuscript preparation without disclosing the sponsor’s role.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID62 = QuestionMultipleChoice(
question_name = "QID62",
question_text = "Submitting research for publication without permission from all authors.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID13 = QuestionLinearScale(
question_name = "QID13",
question_text = "The field of psychology has problems replicating results.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID15 = QuestionLinearScale(
question_name = "QID15",
question_text = "The incentive structure in psychological research can undermine the broader goals of science.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID16 = QuestionLinearScale(
question_name = "QID16",
question_text = "The results from studies with low statistical power are, by definition, incorrect.",
question_options = [1, 2, 3, 4, 5, 6, 7],
option_labels = {1:"Strongly Disagree", 7:"Strongly Agree"}
)
q_QID78 = QuestionFreeText(
question_name = "QID78",
question_text = "This concludes your first session for this research ethics training. Please remember your unique identifier information for the next time you participate in this study. Please click 'Continue' in the lower right-hand corner to provide your email address to receive an Amazon Gift Card. If you do not want to receive a gift card, please close your browser now."
)
q_QID6 = QuestionFreeText(
question_name = "QID6",
question_text = "Timing"
)
q_QID31 = QuestionMultipleChoice(
question_name = "QID31",
question_text = "To enhance chances of publication, violating the ideal of “replace, reduce, refine” regarding the use of research animals.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
# q_QID63 = QuestionMultipleChoice(
# question_name = "QID63",
# question_text = "To enhance chances of publication, violating the ideal of “replace, reduce, refine” regarding the use of research animals.",
# question_options = [
# "Completely Indefensible",
# "Moderately Indefensible",
# "Somewhat Indefensible",
# "Neither Defensible nor Indefensible",
# "Somewhat Defensible",
# "Moderately Defensible",
# "Completely Defensible"
# ]
# )
q_QID30 = QuestionFreeText(
question_name = "QID30",
question_text = "To what extent are each of the following behaviors ethically defensible? Please indicate how defensible you perceive each behavior to be using the scales provided below."
)
# q_QID46 = QuestionFreeText(
# question_name = "QID46",
# question_text = "To what extent are each of the following behaviors ethically defensible? Please indicate how defensible you perceive each behavior to be using the scales provided below."
# )
q_QID49 = QuestionMultipleChoice(
question_name = "QID49",
question_text = "Use of others’ ideas, words, images, or other materials without citation.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID79 = QuestionFreeText(
question_name = "QID79",
question_text = "What is your age in years?"
)
q_QID80 = QuestionMultipleChoice(
question_name = "QID80",
question_text = "What is your gender?",
question_options = [
"Male",
"Female",
"Other"
]
)
q_QID82 = QuestionMultipleChoice(
question_name = "QID82",
question_text = "What is your highest academic degree?",
question_options = [
"B.A.",
"M.A.",
"M.S.",
"M.D.",
"Ph.D.",
"J.D.",
"Other"
]
)
q_QID53 = QuestionMultipleChoice(
question_name = "QID53",
question_text = "Withholding publication specifically to please a sponsor.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
q_QID50 = QuestionMultipleChoice(
question_name = "QID50",
question_text = "Withholding relevant details of methodology or results in publication.",
question_options = [
"Completely Indefensible",
"Moderately Indefensible",
"Somewhat Indefensible",
"Neither Defensible nor Indefensible",
"Somewhat Defensible",
"Moderately Defensible",
"Completely Defensible"
]
)
Combine questions in a Survey¶
This allows you to administer them all at once to a language model. Learn more about adding survey rules and logic (e.g., skip/stop rules, or memories of other questions and answers in a survey).
In [4]:
survey = Survey(questions = [
q_QID44,
# q_QID76,
q_QID32,
# q_QID64,
q_QID56,
q_QID101,
q_QID43,
# q_QID75,
q_QID48,
q_QID35,
# q_QID67,
q_QID40,
# q_QID72,
q_QID38,
# q_QID70,
q_QID55,
q_QID47,
q_QID52,
q_QID36,
# q_QID68,
q_QID21,
q_QID23,
q_QID27,
q_QID28,
q_QID25,
q_QID29,
q_QID22,
q_QID24,
q_QID81,
q_QID90,
q_QID91,
q_QID85,
q_QID98,
q_QID89,
q_QID100,
q_QID92,
q_QID97,
q_QID95,
q_QID86,
q_QID8,
q_QID94,
q_QID88,
q_QID99,
q_QID93,
q_QID96,
q_QID9,
q_QID10,
q_QID11,
q_QID12,
q_QID83,
q_QID84,
q_QID19,
q_QID87,
q_QID59,
q_QID57,
q_QID4,
q_QID60,
q_QID42,
q_QID74,
q_QID20,
q_QID51,
q_QID41,
q_QID73,
q_QID61,
q_QID58,
q_QID14,
q_QID45,
q_QID77,
q_QID17,
q_QID34,
q_QID66,
q_QID39,
# q_QID71,
q_QID37,
# q_QID69,
q_QID33,
# q_QID65,
q_QID18,
q_QID5,
q_QID54,
q_QID62,
q_QID13,
q_QID15,
q_QID16,
q_QID78,
q_QID6,
q_QID31,
# q_QID63,
q_QID30,
# q_QID46,
q_QID49,
q_QID79,
q_QID80,
q_QID82,
q_QID53,
q_QID50
])
Select a language Model to generate responses¶
EDSL works with many popular large language models. To see a list of currently available models:
In [5]:
from edsl import Model
Model.available()
Out[5]:
[['01-ai/Yi-34B-Chat', 'deep_infra', 0], ['Austism/chronos-hermes-13b-v2', 'deep_infra', 1], ['Gryphe/MythoMax-L2-13b', 'deep_infra', 2], ['Gryphe/MythoMax-L2-13b-turbo', 'deep_infra', 3], ['HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1', 'deep_infra', 4], ['Phind/Phind-CodeLlama-34B-v2', 'deep_infra', 5], ['Qwen/Qwen2-72B-Instruct', 'deep_infra', 6], ['Qwen/Qwen2-7B-Instruct', 'deep_infra', 7], ['bigcode/starcoder2-15b', 'deep_infra', 8], ['bigcode/starcoder2-15b-instruct-v0.1', 'deep_infra', 9], ['chatgpt-4o-latest', 'openai', 10], ['claude-3-5-sonnet-20240620', 'anthropic', 11], ['claude-3-haiku-20240307', 'anthropic', 12], ['claude-3-opus-20240229', 'anthropic', 13], ['claude-3-sonnet-20240229', 'anthropic', 14], ['codellama/CodeLlama-34b-Instruct-hf', 'deep_infra', 15], ['codellama/CodeLlama-70b-Instruct-hf', 'deep_infra', 16], ['cognitivecomputations/dolphin-2.6-mixtral-8x7b', 'deep_infra', 17], ['curie:ft-emeritus-2022-11-30-12-58-24', 'openai', 18], ['curie:ft-emeritus-2022-12-01-01-04-36', 'openai', 19], ['curie:ft-emeritus-2022-12-01-01-51-20', 'openai', 20], ['curie:ft-emeritus-2022-12-01-14-16-46', 'openai', 21], ['curie:ft-emeritus-2022-12-01-14-28-00', 'openai', 22], ['curie:ft-emeritus-2022-12-01-14-49-45', 'openai', 23], ['curie:ft-emeritus-2022-12-01-15-29-32', 'openai', 24], ['curie:ft-emeritus-2022-12-01-15-42-25', 'openai', 25], ['curie:ft-emeritus-2022-12-01-15-52-24', 'openai', 26], ['curie:ft-emeritus-2022-12-01-16-40-12', 'openai', 27], ['databricks/dbrx-instruct', 'deep_infra', 28], ['davinci:ft-emeritus-2022-11-30-14-57-33', 'openai', 29], ['deepinfra/airoboros-70b', 'deep_infra', 30], ['gemini-pro', 'google', 31], ['google/codegemma-7b-it', 'deep_infra', 32], ['google/gemma-1.1-7b-it', 'deep_infra', 33], ['gpt-3.5-turbo', 'openai', 34], ['gpt-3.5-turbo-0125', 'openai', 35], ['gpt-3.5-turbo-1106', 'openai', 36], ['gpt-3.5-turbo-16k', 'openai', 37], ['gpt-3.5-turbo-instruct', 'openai', 38], ['gpt-3.5-turbo-instruct-0914', 'openai', 39], ['gpt-4', 'openai', 40], ['gpt-4-0125-preview', 'openai', 41], ['gpt-4-0613', 'openai', 42], ['gpt-4-1106-preview', 'openai', 43], ['gpt-4-turbo', 'openai', 44], ['gpt-4-turbo-2024-04-09', 'openai', 45], ['gpt-4-turbo-preview', 'openai', 46], ['gpt-4o', 'openai', 47], ['gpt-4o-2024-05-13', 'openai', 48], ['gpt-4o-2024-08-06', 'openai', 49], ['gpt-4o-mini', 'openai', 50], ['gpt-4o-mini-2024-07-18', 'openai', 51], ['lizpreciatior/lzlv_70b_fp16_hf', 'deep_infra', 52], ['llava-hf/llava-1.5-7b-hf', 'deep_infra', 53], ['meta-llama/Llama-2-13b-chat-hf', 'deep_infra', 54], ['meta-llama/Llama-2-70b-chat-hf', 'deep_infra', 55], ['meta-llama/Llama-2-7b-chat-hf', 'deep_infra', 56], ['meta-llama/Meta-Llama-3-70B-Instruct', 'deep_infra', 57], ['meta-llama/Meta-Llama-3-8B-Instruct', 'deep_infra', 58], ['microsoft/Phi-3-medium-4k-instruct', 'deep_infra', 59], ['microsoft/WizardLM-2-7B', 'deep_infra', 60], ['microsoft/WizardLM-2-8x22B', 'deep_infra', 61], ['mistralai/Mistral-7B-Instruct-v0.1', 'deep_infra', 62], ['mistralai/Mistral-7B-Instruct-v0.2', 'deep_infra', 63], ['mistralai/Mistral-7B-Instruct-v0.3', 'deep_infra', 64], ['mistralai/Mixtral-8x22B-Instruct-v0.1', 'deep_infra', 65], ['mistralai/Mixtral-8x22B-v0.1', 'deep_infra', 66], ['mistralai/Mixtral-8x7B-Instruct-v0.1', 'deep_infra', 67], ['nvidia/Nemotron-4-340B-Instruct', 'deep_infra', 68], ['openchat/openchat-3.6-8b', 'deep_infra', 69], ['openchat/openchat_3.5', 'deep_infra', 70]]
If we do not specify a language model to use with a survey, GPT 4 preview is used by default. Here we specify a Model for GPT 4 to use with our survey:
In [6]:
model = Model("gpt-4")
In [7]:
results = survey.by(agent).by(model).run()
This generates a formatted dataset of Results that includes information about all of the survey components: questions, scenarios, agents, language models, prompts and responses. To see a list of all components call the columns method:
In [8]:
results.columns
Out[8]:
['agent.age', 'agent.agent_instruction', 'agent.agent_name', 'agent.gender', 'agent.location', 'agent.persona', 'answer.QID10', 'answer.QID100', 'answer.QID101', 'answer.QID11', 'answer.QID12', 'answer.QID13', 'answer.QID14', 'answer.QID15', 'answer.QID16', 'answer.QID17', 'answer.QID18', 'answer.QID19', 'answer.QID20', 'answer.QID21', 'answer.QID22', 'answer.QID23', 'answer.QID24', 'answer.QID25', 'answer.QID27', 'answer.QID28', 'answer.QID29', 'answer.QID30', 'answer.QID31', 'answer.QID32', 'answer.QID33', 'answer.QID34', 'answer.QID35', 'answer.QID36', 'answer.QID37', 'answer.QID38', 'answer.QID39', 'answer.QID4', 'answer.QID40', 'answer.QID41', 'answer.QID42', 'answer.QID43', 'answer.QID44', 'answer.QID45', 'answer.QID47', 'answer.QID48', 'answer.QID49', 'answer.QID5', 'answer.QID50', 'answer.QID51', 'answer.QID52', 'answer.QID53', 'answer.QID54', 'answer.QID55', 'answer.QID56', 'answer.QID57', 'answer.QID58', 'answer.QID59', 'answer.QID6', 'answer.QID60', 'answer.QID61', 'answer.QID62', 'answer.QID66', 'answer.QID73', 'answer.QID74', 'answer.QID77', 'answer.QID78', 'answer.QID79', 'answer.QID8', 'answer.QID80', 'answer.QID81', 'answer.QID82', 'answer.QID83', 'answer.QID84', 'answer.QID85', 'answer.QID86', 'answer.QID87', 'answer.QID88', 'answer.QID89', 'answer.QID9', 'answer.QID90', 'answer.QID91', 'answer.QID92', 'answer.QID93', 'answer.QID94', 'answer.QID95', 'answer.QID96', 'answer.QID97', 'answer.QID98', 'answer.QID99', 'comment.QID100_comment', 'comment.QID10_comment', 'comment.QID11_comment', 'comment.QID12_comment', 'comment.QID13_comment', 'comment.QID14_comment', 'comment.QID15_comment', 'comment.QID16_comment', 'comment.QID17_comment', 'comment.QID18_comment', 'comment.QID19_comment', 'comment.QID20_comment', 'comment.QID21_comment', 'comment.QID22_comment', 'comment.QID23_comment', 'comment.QID24_comment', 'comment.QID25_comment', 'comment.QID27_comment', 'comment.QID28_comment', 'comment.QID29_comment', 'comment.QID30_comment', 'comment.QID31_comment', 'comment.QID32_comment', 'comment.QID33_comment', 'comment.QID34_comment', 'comment.QID35_comment', 'comment.QID36_comment', 'comment.QID37_comment', 'comment.QID38_comment', 'comment.QID39_comment', 'comment.QID40_comment', 'comment.QID41_comment', 'comment.QID42_comment', 'comment.QID43_comment', 'comment.QID44_comment', 'comment.QID45_comment', 'comment.QID47_comment', 'comment.QID48_comment', 'comment.QID49_comment', 'comment.QID50_comment', 'comment.QID51_comment', 'comment.QID52_comment', 'comment.QID53_comment', 'comment.QID54_comment', 'comment.QID55_comment', 'comment.QID56_comment', 'comment.QID57_comment', 'comment.QID58_comment', 'comment.QID59_comment', 'comment.QID60_comment', 'comment.QID61_comment', 'comment.QID62_comment', 'comment.QID66_comment', 'comment.QID6_comment', 'comment.QID73_comment', 'comment.QID74_comment', 'comment.QID77_comment', 'comment.QID80_comment', 'comment.QID81_comment', 'comment.QID82_comment', 'comment.QID83_comment', 'comment.QID84_comment', 'comment.QID85_comment', 'comment.QID86_comment', 'comment.QID87_comment', 'comment.QID88_comment', 'comment.QID89_comment', 'comment.QID8_comment', 'comment.QID90_comment', 'comment.QID91_comment', 'comment.QID92_comment', 'comment.QID93_comment', 'comment.QID94_comment', 'comment.QID95_comment', 'comment.QID96_comment', 'comment.QID97_comment', 'comment.QID98_comment', 'comment.QID99_comment', 'comment.QID9_comment', 'model.frequency_penalty', 'model.logprobs', 'model.max_tokens', 'model.model', 'model.presence_penalty', 'model.temperature', 'model.top_logprobs', 'model.top_p', 'prompt.QID100_system_prompt', 'prompt.QID100_user_prompt', 'prompt.QID101_system_prompt', 'prompt.QID101_user_prompt', 'prompt.QID10_system_prompt', 'prompt.QID10_user_prompt', 'prompt.QID11_system_prompt', 'prompt.QID11_user_prompt', 'prompt.QID12_system_prompt', 'prompt.QID12_user_prompt', 'prompt.QID13_system_prompt', 'prompt.QID13_user_prompt', 'prompt.QID14_system_prompt', 'prompt.QID14_user_prompt', 'prompt.QID15_system_prompt', 'prompt.QID15_user_prompt', 'prompt.QID16_system_prompt', 'prompt.QID16_user_prompt', 'prompt.QID17_system_prompt', 'prompt.QID17_user_prompt', 'prompt.QID18_system_prompt', 'prompt.QID18_user_prompt', 'prompt.QID19_system_prompt', 'prompt.QID19_user_prompt', 'prompt.QID20_system_prompt', 'prompt.QID20_user_prompt', 'prompt.QID21_system_prompt', 'prompt.QID21_user_prompt', 'prompt.QID22_system_prompt', 'prompt.QID22_user_prompt', 'prompt.QID23_system_prompt', 'prompt.QID23_user_prompt', 'prompt.QID24_system_prompt', 'prompt.QID24_user_prompt', 'prompt.QID25_system_prompt', 'prompt.QID25_user_prompt', 'prompt.QID27_system_prompt', 'prompt.QID27_user_prompt', 'prompt.QID28_system_prompt', 'prompt.QID28_user_prompt', 'prompt.QID29_system_prompt', 'prompt.QID29_user_prompt', 'prompt.QID30_system_prompt', 'prompt.QID30_user_prompt', 'prompt.QID31_system_prompt', 'prompt.QID31_user_prompt', 'prompt.QID32_system_prompt', 'prompt.QID32_user_prompt', 'prompt.QID33_system_prompt', 'prompt.QID33_user_prompt', 'prompt.QID34_system_prompt', 'prompt.QID34_user_prompt', 'prompt.QID35_system_prompt', 'prompt.QID35_user_prompt', 'prompt.QID36_system_prompt', 'prompt.QID36_user_prompt', 'prompt.QID37_system_prompt', 'prompt.QID37_user_prompt', 'prompt.QID38_system_prompt', 'prompt.QID38_user_prompt', 'prompt.QID39_system_prompt', 'prompt.QID39_user_prompt', 'prompt.QID40_system_prompt', 'prompt.QID40_user_prompt', 'prompt.QID41_system_prompt', 'prompt.QID41_user_prompt', 'prompt.QID42_system_prompt', 'prompt.QID42_user_prompt', 'prompt.QID43_system_prompt', 'prompt.QID43_user_prompt', 'prompt.QID44_system_prompt', 'prompt.QID44_user_prompt', 'prompt.QID45_system_prompt', 'prompt.QID45_user_prompt', 'prompt.QID47_system_prompt', 'prompt.QID47_user_prompt', 'prompt.QID48_system_prompt', 'prompt.QID48_user_prompt', 'prompt.QID49_system_prompt', 'prompt.QID49_user_prompt', 'prompt.QID4_system_prompt', 'prompt.QID4_user_prompt', 'prompt.QID50_system_prompt', 'prompt.QID50_user_prompt', 'prompt.QID51_system_prompt', 'prompt.QID51_user_prompt', 'prompt.QID52_system_prompt', 'prompt.QID52_user_prompt', 'prompt.QID53_system_prompt', 'prompt.QID53_user_prompt', 'prompt.QID54_system_prompt', 'prompt.QID54_user_prompt', 'prompt.QID55_system_prompt', 'prompt.QID55_user_prompt', 'prompt.QID56_system_prompt', 'prompt.QID56_user_prompt', 'prompt.QID57_system_prompt', 'prompt.QID57_user_prompt', 'prompt.QID58_system_prompt', 'prompt.QID58_user_prompt', 'prompt.QID59_system_prompt', 'prompt.QID59_user_prompt', 'prompt.QID5_system_prompt', 'prompt.QID5_user_prompt', 'prompt.QID60_system_prompt', 'prompt.QID60_user_prompt', 'prompt.QID61_system_prompt', 'prompt.QID61_user_prompt', 'prompt.QID62_system_prompt', 'prompt.QID62_user_prompt', 'prompt.QID66_system_prompt', 'prompt.QID66_user_prompt', 'prompt.QID6_system_prompt', 'prompt.QID6_user_prompt', 'prompt.QID73_system_prompt', 'prompt.QID73_user_prompt', 'prompt.QID74_system_prompt', 'prompt.QID74_user_prompt', 'prompt.QID77_system_prompt', 'prompt.QID77_user_prompt', 'prompt.QID78_system_prompt', 'prompt.QID78_user_prompt', 'prompt.QID79_system_prompt', 'prompt.QID79_user_prompt', 'prompt.QID80_system_prompt', 'prompt.QID80_user_prompt', 'prompt.QID81_system_prompt', 'prompt.QID81_user_prompt', 'prompt.QID82_system_prompt', 'prompt.QID82_user_prompt', 'prompt.QID83_system_prompt', 'prompt.QID83_user_prompt', 'prompt.QID84_system_prompt', 'prompt.QID84_user_prompt', 'prompt.QID85_system_prompt', 'prompt.QID85_user_prompt', 'prompt.QID86_system_prompt', 'prompt.QID86_user_prompt', 'prompt.QID87_system_prompt', 'prompt.QID87_user_prompt', 'prompt.QID88_system_prompt', 'prompt.QID88_user_prompt', 'prompt.QID89_system_prompt', 'prompt.QID89_user_prompt', 'prompt.QID8_system_prompt', 'prompt.QID8_user_prompt', 'prompt.QID90_system_prompt', 'prompt.QID90_user_prompt', 'prompt.QID91_system_prompt', 'prompt.QID91_user_prompt', 'prompt.QID92_system_prompt', 'prompt.QID92_user_prompt', 'prompt.QID93_system_prompt', 'prompt.QID93_user_prompt', 'prompt.QID94_system_prompt', 'prompt.QID94_user_prompt', 'prompt.QID95_system_prompt', 'prompt.QID95_user_prompt', 'prompt.QID96_system_prompt', 'prompt.QID96_user_prompt', 'prompt.QID97_system_prompt', 'prompt.QID97_user_prompt', 'prompt.QID98_system_prompt', 'prompt.QID98_user_prompt', 'prompt.QID99_system_prompt', 'prompt.QID99_user_prompt', 'prompt.QID9_system_prompt', 'prompt.QID9_user_prompt', 'question_options.QID100_question_options', 'question_options.QID101_question_options', 'question_options.QID10_question_options', 'question_options.QID11_question_options', 'question_options.QID12_question_options', 'question_options.QID13_question_options', 'question_options.QID14_question_options', 'question_options.QID15_question_options', 'question_options.QID16_question_options', 'question_options.QID17_question_options', 'question_options.QID18_question_options', 'question_options.QID19_question_options', 'question_options.QID20_question_options', 'question_options.QID21_question_options', 'question_options.QID22_question_options', 'question_options.QID23_question_options', 'question_options.QID24_question_options', 'question_options.QID25_question_options', 'question_options.QID27_question_options', 'question_options.QID28_question_options', 'question_options.QID29_question_options', 'question_options.QID30_question_options', 'question_options.QID31_question_options', 'question_options.QID32_question_options', 'question_options.QID33_question_options', 'question_options.QID34_question_options', 'question_options.QID35_question_options', 'question_options.QID36_question_options', 'question_options.QID37_question_options', 'question_options.QID38_question_options', 'question_options.QID39_question_options', 'question_options.QID40_question_options', 'question_options.QID41_question_options', 'question_options.QID42_question_options', 'question_options.QID43_question_options', 'question_options.QID44_question_options', 'question_options.QID45_question_options', 'question_options.QID47_question_options', 'question_options.QID48_question_options', 'question_options.QID49_question_options', 'question_options.QID4_question_options', 'question_options.QID50_question_options', 'question_options.QID51_question_options', 'question_options.QID52_question_options', 'question_options.QID53_question_options', 'question_options.QID54_question_options', 'question_options.QID55_question_options', 'question_options.QID56_question_options', 'question_options.QID57_question_options', 'question_options.QID58_question_options', 'question_options.QID59_question_options', 'question_options.QID5_question_options', 'question_options.QID60_question_options', 'question_options.QID61_question_options', 'question_options.QID62_question_options', 'question_options.QID66_question_options', 'question_options.QID6_question_options', 'question_options.QID73_question_options', 'question_options.QID74_question_options', 'question_options.QID77_question_options', 'question_options.QID78_question_options', 'question_options.QID79_question_options', 'question_options.QID80_question_options', 'question_options.QID81_question_options', 'question_options.QID82_question_options', 'question_options.QID83_question_options', 'question_options.QID84_question_options', 'question_options.QID85_question_options', 'question_options.QID86_question_options', 'question_options.QID87_question_options', 'question_options.QID88_question_options', 'question_options.QID89_question_options', 'question_options.QID8_question_options', 'question_options.QID90_question_options', 'question_options.QID91_question_options', 'question_options.QID92_question_options', 'question_options.QID93_question_options', 'question_options.QID94_question_options', 'question_options.QID95_question_options', 'question_options.QID96_question_options', 'question_options.QID97_question_options', 'question_options.QID98_question_options', 'question_options.QID99_question_options', 'question_options.QID9_question_options', 'question_text.QID100_question_text', 'question_text.QID101_question_text', 'question_text.QID10_question_text', 'question_text.QID11_question_text', 'question_text.QID12_question_text', 'question_text.QID13_question_text', 'question_text.QID14_question_text', 'question_text.QID15_question_text', 'question_text.QID16_question_text', 'question_text.QID17_question_text', 'question_text.QID18_question_text', 'question_text.QID19_question_text', 'question_text.QID20_question_text', 'question_text.QID21_question_text', 'question_text.QID22_question_text', 'question_text.QID23_question_text', 'question_text.QID24_question_text', 'question_text.QID25_question_text', 'question_text.QID27_question_text', 'question_text.QID28_question_text', 'question_text.QID29_question_text', 'question_text.QID30_question_text', 'question_text.QID31_question_text', 'question_text.QID32_question_text', 'question_text.QID33_question_text', 'question_text.QID34_question_text', 'question_text.QID35_question_text', 'question_text.QID36_question_text', 'question_text.QID37_question_text', 'question_text.QID38_question_text', 'question_text.QID39_question_text', 'question_text.QID40_question_text', 'question_text.QID41_question_text', 'question_text.QID42_question_text', 'question_text.QID43_question_text', 'question_text.QID44_question_text', 'question_text.QID45_question_text', 'question_text.QID47_question_text', 'question_text.QID48_question_text', 'question_text.QID49_question_text', 'question_text.QID4_question_text', 'question_text.QID50_question_text', 'question_text.QID51_question_text', 'question_text.QID52_question_text', 'question_text.QID53_question_text', 'question_text.QID54_question_text', 'question_text.QID55_question_text', 'question_text.QID56_question_text', 'question_text.QID57_question_text', 'question_text.QID58_question_text', 'question_text.QID59_question_text', 'question_text.QID5_question_text', 'question_text.QID60_question_text', 'question_text.QID61_question_text', 'question_text.QID62_question_text', 'question_text.QID66_question_text', 'question_text.QID6_question_text', 'question_text.QID73_question_text', 'question_text.QID74_question_text', 'question_text.QID77_question_text', 'question_text.QID78_question_text', 'question_text.QID79_question_text', 'question_text.QID80_question_text', 'question_text.QID81_question_text', 'question_text.QID82_question_text', 'question_text.QID83_question_text', 'question_text.QID84_question_text', 'question_text.QID85_question_text', 'question_text.QID86_question_text', 'question_text.QID87_question_text', 'question_text.QID88_question_text', 'question_text.QID89_question_text', 'question_text.QID8_question_text', 'question_text.QID90_question_text', 'question_text.QID91_question_text', 'question_text.QID92_question_text', 'question_text.QID93_question_text', 'question_text.QID94_question_text', 'question_text.QID95_question_text', 'question_text.QID96_question_text', 'question_text.QID97_question_text', 'question_text.QID98_question_text', 'question_text.QID99_question_text', 'question_text.QID9_question_text', 'question_type.QID100_question_type', 'question_type.QID101_question_type', 'question_type.QID10_question_type', 'question_type.QID11_question_type', 'question_type.QID12_question_type', 'question_type.QID13_question_type', 'question_type.QID14_question_type', 'question_type.QID15_question_type', 'question_type.QID16_question_type', 'question_type.QID17_question_type', 'question_type.QID18_question_type', 'question_type.QID19_question_type', 'question_type.QID20_question_type', 'question_type.QID21_question_type', 'question_type.QID22_question_type', 'question_type.QID23_question_type', 'question_type.QID24_question_type', 'question_type.QID25_question_type', 'question_type.QID27_question_type', 'question_type.QID28_question_type', 'question_type.QID29_question_type', 'question_type.QID30_question_type', 'question_type.QID31_question_type', 'question_type.QID32_question_type', 'question_type.QID33_question_type', 'question_type.QID34_question_type', 'question_type.QID35_question_type', 'question_type.QID36_question_type', 'question_type.QID37_question_type', 'question_type.QID38_question_type', 'question_type.QID39_question_type', 'question_type.QID40_question_type', 'question_type.QID41_question_type', 'question_type.QID42_question_type', 'question_type.QID43_question_type', 'question_type.QID44_question_type', 'question_type.QID45_question_type', 'question_type.QID47_question_type', 'question_type.QID48_question_type', 'question_type.QID49_question_type', 'question_type.QID4_question_type', 'question_type.QID50_question_type', 'question_type.QID51_question_type', 'question_type.QID52_question_type', 'question_type.QID53_question_type', 'question_type.QID54_question_type', 'question_type.QID55_question_type', 'question_type.QID56_question_type', 'question_type.QID57_question_type', 'question_type.QID58_question_type', 'question_type.QID59_question_type', 'question_type.QID5_question_type', 'question_type.QID60_question_type', 'question_type.QID61_question_type', 'question_type.QID62_question_type', 'question_type.QID66_question_type', 'question_type.QID6_question_type', 'question_type.QID73_question_type', 'question_type.QID74_question_type', 'question_type.QID77_question_type', 'question_type.QID78_question_type', 'question_type.QID79_question_type', 'question_type.QID80_question_type', 'question_type.QID81_question_type', 'question_type.QID82_question_type', 'question_type.QID83_question_type', 'question_type.QID84_question_type', 'question_type.QID85_question_type', 'question_type.QID86_question_type', 'question_type.QID87_question_type', 'question_type.QID88_question_type', 'question_type.QID89_question_type', 'question_type.QID8_question_type', 'question_type.QID90_question_type', 'question_type.QID91_question_type', 'question_type.QID92_question_type', 'question_type.QID93_question_type', 'question_type.QID94_question_type', 'question_type.QID95_question_type', 'question_type.QID96_question_type', 'question_type.QID97_question_type', 'question_type.QID98_question_type', 'question_type.QID99_question_type', 'question_type.QID9_question_type', 'raw_model_response.QID100_raw_model_response', 'raw_model_response.QID101_raw_model_response', 'raw_model_response.QID10_raw_model_response', 'raw_model_response.QID11_raw_model_response', 'raw_model_response.QID12_raw_model_response', 'raw_model_response.QID13_raw_model_response', 'raw_model_response.QID14_raw_model_response', 'raw_model_response.QID15_raw_model_response', 'raw_model_response.QID16_raw_model_response', 'raw_model_response.QID17_raw_model_response', 'raw_model_response.QID18_raw_model_response', 'raw_model_response.QID19_raw_model_response', 'raw_model_response.QID20_raw_model_response', 'raw_model_response.QID21_raw_model_response', 'raw_model_response.QID22_raw_model_response', 'raw_model_response.QID23_raw_model_response', 'raw_model_response.QID24_raw_model_response', 'raw_model_response.QID25_raw_model_response', 'raw_model_response.QID27_raw_model_response', 'raw_model_response.QID28_raw_model_response', 'raw_model_response.QID29_raw_model_response', 'raw_model_response.QID30_raw_model_response', 'raw_model_response.QID31_raw_model_response', 'raw_model_response.QID32_raw_model_response', 'raw_model_response.QID33_raw_model_response', 'raw_model_response.QID34_raw_model_response', 'raw_model_response.QID35_raw_model_response', 'raw_model_response.QID36_raw_model_response', 'raw_model_response.QID37_raw_model_response', 'raw_model_response.QID38_raw_model_response', 'raw_model_response.QID39_raw_model_response', 'raw_model_response.QID40_raw_model_response', 'raw_model_response.QID41_raw_model_response', 'raw_model_response.QID42_raw_model_response', 'raw_model_response.QID43_raw_model_response', 'raw_model_response.QID44_raw_model_response', 'raw_model_response.QID45_raw_model_response', 'raw_model_response.QID47_raw_model_response', 'raw_model_response.QID48_raw_model_response', 'raw_model_response.QID49_raw_model_response', 'raw_model_response.QID4_raw_model_response', 'raw_model_response.QID50_raw_model_response', 'raw_model_response.QID51_raw_model_response', 'raw_model_response.QID52_raw_model_response', 'raw_model_response.QID53_raw_model_response', 'raw_model_response.QID54_raw_model_response', 'raw_model_response.QID55_raw_model_response', 'raw_model_response.QID56_raw_model_response', 'raw_model_response.QID57_raw_model_response', 'raw_model_response.QID58_raw_model_response', 'raw_model_response.QID59_raw_model_response', 'raw_model_response.QID5_raw_model_response', 'raw_model_response.QID60_raw_model_response', 'raw_model_response.QID61_raw_model_response', 'raw_model_response.QID62_raw_model_response', 'raw_model_response.QID66_raw_model_response', 'raw_model_response.QID6_raw_model_response', 'raw_model_response.QID73_raw_model_response', 'raw_model_response.QID74_raw_model_response', 'raw_model_response.QID77_raw_model_response', 'raw_model_response.QID78_raw_model_response', 'raw_model_response.QID79_raw_model_response', 'raw_model_response.QID80_raw_model_response', 'raw_model_response.QID81_raw_model_response', 'raw_model_response.QID82_raw_model_response', 'raw_model_response.QID83_raw_model_response', 'raw_model_response.QID84_raw_model_response', 'raw_model_response.QID85_raw_model_response', 'raw_model_response.QID86_raw_model_response', 'raw_model_response.QID87_raw_model_response', 'raw_model_response.QID88_raw_model_response', 'raw_model_response.QID89_raw_model_response', 'raw_model_response.QID8_raw_model_response', 'raw_model_response.QID90_raw_model_response', 'raw_model_response.QID91_raw_model_response', 'raw_model_response.QID92_raw_model_response', 'raw_model_response.QID93_raw_model_response', 'raw_model_response.QID94_raw_model_response', 'raw_model_response.QID95_raw_model_response', 'raw_model_response.QID96_raw_model_response', 'raw_model_response.QID97_raw_model_response', 'raw_model_response.QID98_raw_model_response', 'raw_model_response.QID99_raw_model_response', 'raw_model_response.QID9_raw_model_response']
Working with results¶
We can use built-in methods for analyzing results:
In [9]:
(results
.select("persona", "QID44", "QID32", "QID56")
.print(format="rich",
pretty_labels = {
"agent.persona":"Persona",
"answer.QID44":q_QID44.question_text,
"answer.QID32":q_QID32.question_text,
"answer.QID56":q_QID56.question_text
})
)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ Adding additional research ┃ ┃ ┃ ┃ Acknowledging another's ┃ participants because the ┃ ┃ ┃ ┃ technical assistance in ┃ results collected thus far ┃ Adding or deleting ┃ ┃ ┃ publication without that ┃ are not yet statistically ┃ co-authors to increase ┃ ┃ Persona ┃ person's permission. ┃ significant. ┃ chances of acceptance. ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ You are a graduate student │ Somewhat Defensible │ Moderately Indefensible │ Completely Indefensible │ │ in the social sciences. │ │ │ │ └────────────────────────────┴───────────────────────────┴────────────────────────────┴───────────────────────────┘
Post content to the Coop¶
Coop is a new platform for creating, storing and sharing LLM-based research using EDSL. Learn more about using the Coop to facilitate your research.
Here we post the survey, results and this notebook. We can optionally pass a description and visibility status (default is unlisted):
In [10]:
survey.push(description = "Example OSF survey", visibility = "public")
Out[10]:
{'description': 'Example OSF survey',
'object_type': 'survey',
'url': 'https://www.expectedparrot.com/content/17533592-7c67-4788-88d5-84cdc5f655ca',
'uuid': '17533592-7c67-4788-88d5-84cdc5f655ca',
'version': '0.1.31.dev4',
'visibility': 'public'}
In [11]:
results.push(description = "Example results of OSF survey", visibility = "public")
Out[11]:
{'description': 'Example results of OSF survey',
'object_type': 'results',
'url': 'https://www.expectedparrot.com/content/040bdcb9-0168-4713-8ed0-e582f82dde0d',
'uuid': '040bdcb9-0168-4713-8ed0-e582f82dde0d',
'version': '0.1.31.dev4',
'visibility': 'public'}
In [1]:
from edsl import Notebook
n = Notebook(path = "osf_example_survey.ipynb")
n.push(description = "OSF example survey", visibility = "public")
Out[1]:
{'description': 'OSF example survey',
'object_type': 'notebook',
'url': 'https://www.expectedparrot.com/content/f865ec42-4294-417c-8f75-40d528abf1fe',
'uuid': 'f865ec42-4294-417c-8f75-40d528abf1fe',
'version': '0.1.31.dev4',
'visibility': 'public'}
To update content at the Coop:
In [13]:
from edsl import Notebook
n = Notebook(path = "osf_example_survey.ipynb")
n.patch(uuid = "f865ec42-4294-417c-8f75-40d528abf1fe", value = n)
Out[13]:
{'status': 'success'}
from edsl import Notebook
notebook = Notebook.pull("f865ec42-4294-417c-8f75-40d528abf1fe")
# this object is too large to display here