WHAT IS RESEARCH? * Book sez: "The systematic and objective process of gathering, recording, and analyzing data for aid in making decisions" * Types of research Exploratory - Find the problem Descriptive - Describe the problem H+4W Causal - Investigate the solutions/reasons Exploratory Research -- Is there anything going on? Highest risk, least structured Usually fishing expeditions -- Is anything there? Gather anything that may be pertinent Often get nothing, tough to justify the next one Most fun, least precise Descriptive Research -- Find out specifics? Who, what, where, when, how Usually accompanied by descriptive statistics Tabulate, classify, categorize Finite number of classes Reports facts Causal Research Attempts to identify cause and effect Frequently only correlation (only God knows the cause) Most often done in academic/scientific settings - Frequently bogus (paid to find cancer) - Usually experimental methods - must be careful to analyze random components Book suggests 1. Establish appropriate causal order/sequence 2. Measure variation between "cause" and "effect" 3. Recognize presence or absence of alternative explanations When is research justified? When we want to question assumptions When we have the time to do it -- good planning When we have the information or opportunity When the results will influence the decision to be made When we can justify the costs with the benefits THE RESEARCH PROCESS The basic stages of a research project: Defining the research question/problem May be the most important step Often the most neglected Too broad - lose focus Too narrow - minimum benefits for setup cost Planning the research design Good question will lend itself to more than one approach Theoretical foundation - what hypotheses will be tested - what are the "expected" results - implications of unexpected results Many ways to answer the question * Experiments - Laboratory - Field * Survey - Cross sectional - Longitudinal - Focus or Delphi groups * Observation - Time study - Work sampling - Physiology * Computer Modeling - simulation - linear/math programming Defining the sampling plan and/or information domain What is the "population" to be used How will we select representative members How many do we need Where will we look for prior work, secondary data Collecting data/running experiments Where can errors creep in How can human factors muck it up How can we control the situation When do we have enough data How do we flag bad data How do we handle unanticipated data, responses, reactions Analyzing the results What methods will we use? - Descriptive - Multi variate - Non parametric What conclusions can we draw? How strong are the conclusions How do we explain unanticipated results? Preparing the report Who is the audience? What vehicles best display the results? - graphs - charts - tables - models How do I answer the criticism - questions I didn't answer - alternative explanations - irrational arguments How do I organize it to make the best case DEFINING THE RESEARCH QUESTION This is potentially the toughest and most critical stage of the entire research process! Provides focus of attention and goal "What are we trying to accomplish with the research?" Must look forward - potential information source/subject pool - theoretical basis - hypotheses - outline of research design Must look backward - may not be able to answer - may not really meet business objectives Must start with the universe and throw away least plausible alternatives Written statement, in English, forces focus and invites criticism when it is cheapest and most valuable -- before the work begins Poor statement usually wastes a lot of time and effort RESEARCHER'S TRUTHS You will never wind up where you intended to go. You will never find a perfect way to answer the question. There will always be a question you didn't answer. No matter how much time you spend planning things, you will always forget something. Never underestimate the ability of human beings to foul up your plans The world is irrational, unfair and Murphy runs it. Your client's first response will be "I don't know what I want but I know that isn't it?" As soon as you lock on an approach, you have bypassed a better one. If was easy, the Big Kahuna would do it. The question is always too big to answer. THE "SCIENTIFIC METHOD" AND OTHER MYTHS Review and assess all relevant knowledge Literature search - find "everything" Don't want to do anything that's been done before Formulate definitions and theory Need clear definitions of all terms Theory - suggests how one thing will affect another "A rise in income level will result in a rise in spending" State hypotheses (straw man) Best to state the negative of our theory (e.g. "Changes in income level will have no effect on spending") We can prove that a statement is false, but very difficult to prove that any statement is always true. Design empirical tests for hypothesis Propose variables for manipulation - Independent -- those you plan to manipulate (income level) - Dependent -- those you plan to measure (spending) Anticipate possible outcomes Decide on how you will account for alternative explanations Decide how you will isolate only the factors you manipulate Decide on how certain you must be to come to a conclusion Acquire results from empirical tests What data gathering mechanisms will you use How will you control for errors How long will you wait Evaluate the results What methods will you use Are all data valid Are there alternative tests/explanations Explain the results, and identify the new question Good research generates more questions than it answers New theories to explain unanticipated results Internal validity - are there any holes in your work - flaws in design that may explain results - other variables that may have affected results External validity (never can address) - will it work the same all the time - how extendable are results EXPLORATORY RESEARCH Usually done to help define another research question "Universe" of discovery -- What's out there? Select from alternatives -- Lack 'o bucks Primary Methods Individual and group interviews -- Focus or "Delphi" methods Secondary Data analysis - "Data Grubbing" INTERVIEWS WITH "EXPERTS" -- May be persons knowledgeable in field or "target" population Experience surveys -- "You know all about this stuff" - Often informal - Small group - 1 to 5 - Use to challenge a hypothesis - Upside + cheap can be "cocktail" chatter + may save time "barking up the wrong tree" + politically astute - line up supporters - Downside - experts not always right (Sony & tape recorder) - non scientific (not random or structured) - may institutionalize bad assumptions Focus groups-Delphi - Orchestrated discussion for selected "target" consumers/users - Screen/refine concepts or products - 8 - 12 people - smaller too much individual influence - bigger too unwieldy to control Upside + Fast and easy + Inexpensive + Easy to Analyze + synergy -- Group total > {SUM} of Individuals Downside - qualitative only - not necessarily representative of larger population - may be overly influenced by leader qualities/skills - may be dominated by strong personality Case Study Depth Interview EXPLORATORY RESEARCH Usually done to help define another research question "Universe" of discovery -- What's out there? Select from alternatives -- Lack 'o bucks Primary Methods Individual and group interviews -- Focus or "Delphi" methods Secondary Data analysis - "Data Grubbing" (tomorrow) INTERVIEWS WITH "EXPERTS" -- May be persons knowledgeable in field or "target" population EXPERIENCE SURVEYS -- "You know all about this stuff" Often informal Small group - 1 to 5 Use to challenge a hypothesis Upside + cheap can be "cocktail" chatter + may save time "barking up the wrong tree" + politically astute - line up supporters Downside - experts not always right (Sony & tape recorder) - non scientific (not random or structured) - may institutionalize bad assumptions FOCUS GROUPS - Delphi Methods VERY HOT! Orchestrated discussion for selected "target" consumers/users Screen/refine concepts or products 8 - 12 people - smaller too much individual influence - bigger too unwieldy to control Upside + Fast and easy to setup + Inexpensive + Easy to Analyze -- Formal and "hidden" results (videotape?) + Synergy -- Group total > {SUM} of Individuals + Group dynamics -- snowball, security, stimulation + Structure advantage Downside - qualitative only - requires expediter - not necessarily representative of larger population - may be overly influenced by leader qualities/skills - may be dominated by strong personality - environmental influence CASE STUDY -- "My thing is like your thing" Essentially research by analogy Usually in depth Upside + More complete picture of problem + Triggers things you forgot + Unbiased researcher gathering facts Downside - Requires cooperation of the "studied" - Assumes honesty by the studied - External validity questions "Yeah, but what about here" - Needs careful identification of underlying conditions DEPTH INTERVIEWS AND PROJECTIVE TECHNIQUES Essentially psychology and perception - ink blots, word association Marketers know people buy what they want -- not what they need Attempts to deal with irrationality of the world Upside + recognizes irrationality + may find things even subjects didn't know (Betty Crocker) + probably better at predicting what people will really do Downside - requires some real expertise and sensitivity - subjects may still lie - who does the work may be important (social facilitation) - better be really careful with ethical implications SURVEYS A PRIMARY DATA COLLECTION METHOD BASED ON COMMUNICATION WITH A REPRESENTATIVE SAMPLE OF INDIVIDUALS. UPSIDE + Relatively easy to do -- minimal expense + Efficient -- cover a lot of info with minimal hassle + Flexible -- many permutations DOWNSIDE - People lie - Answers may be influenced by questions, persons, environment - Hard to control bias - Subject to many potential errors SOME CLASSIFICATIONS FOR SURVEYS ~ Cross-Sectional -- Many respondents, same time "How does everybody feel?" ~ Longitudinal -- Same respondents over a longer time " Do you still feel that way?" ERRORS - The curse of the survey ~ RANDOM - Nothing you can do, stuff happens ~ SYSTEMATIC - Regular, something you can reduce TYPES OF SYSTEMATIC ERRORS (BIAS) * Researcher's -- You did it Data Collection and Transformation - "keypunch" - transposition/reading - lost/mangled questionnaires - handwriting - multiple coders - mistakes on questionnaire fill-out Sampling - non random sample - poor blocking - no backup - lack of balance - poor execution Interviewers/analysts - cheat on requirements (fill out themselves) - "cook" the data to achieve results (will find cancer) Respondent's -- They did it May be deliberate or unconscious Self selection -- When response is voluntary, those who respond are usually statistically different then those who do not. Often the respondents with extreme views: - ax to grind on "quality of service" - delighted with service - can't be bothered - want to screw up results - hate surveys - interested in results Lies -- deliberately deceives you - what do you expect them to say - what would their peers expect them to say - like/don't like you or your firm Unconscious Misrepresentation - Acquiescence -- Some people tend to like (or agree with) every thing they hear for the first time - Extremity -- Others are just contrary, or tend to see the world as black and white. They will opt for polar or shocking responses - - Social Facilitation -- The unconscious cues (body language) or the mere presence of another human being may alter the way the respondent answers. Don't want to appear stupid, like your bod, etc. - Auspices -- Who is asking may bias the results. - Social Desirability -- Estimates may be biased what people feel they should be doing (or shouldn't be doing) or by their own failures or inadequacies. (How much do you earn?) - Misunderstanding the question -- People may not know what you are asking - Primacy/Recency -- Estimates of factors may be influenced by how recently the activity was done or how important the respondent considers the activity to be. TYPES OF SURVEYS * PERSONAL INTERVIEWS - Person to Person Best when you want to minimize nonresponse Upside + Allows clarification of questions + Gather impressions + Minimizes self selection + Immediate feed back + Props/visual aids Downside - Highest cost/respondent - Interviewer influence - Differences in interviewers (if more than one) - No anonymity Door to Door - You go to them + Their turf + You determine the sampling - You travel - What to do when not there Mall Intercept + cheap - respondents come to you + efficient - many people in minimal time + may offer samples - always biased - no demographic controls TELEPHONE INTERVIEWS Best for lots of detail and personal information Upside + relatively cheap + minimal interviewer influence, less of a threat + very fast + availability of canned phone lists Downside - may have to go commercial to get demographics - respondent can't see what's going on & vice versa - requires more structured questions - easier to blow off the interviewer - unlisted phone numbers SELF ADMINISTERED QUESTIONNAIRES Best for in depth quality of response and very sensitive questions Upside + respondent convenience - fill out when they want to + very cheap - minimal human resources + geography not a problem + availability of mailing lists + anonymity Downside - once gone, no control - "no response" very likely - bias in non response - no explanatory feedback - requires structured questions - must be right the first time--tough to change - timeliness of responses QUESTIONNAIRES TYPES OF QUESTIONS * OPEN ENDED ~ Best for exploratory use ~ Range not known ~ Tough to apply quantitative methods ~ Good for anecdotal information ~ Good first question * STRUCTURED Scaled Simple Attitude - series of YES/NO questions with a common descriptor - How would you describe married life? Delirious YES NO ? Satisfying YES NO ? Frustrating YES NO ? - Good for distinguishing nuances of impressions Category choices - e.g. Never, Rarely, Occasionally, Often, Always - e.g. Less Than 1 year, 1 to 5 years, More than 5 years - report % choosing an option, above/below the average Summated Ratings (Likert Scale) - Agree/disagree with a factual statement - Can assign some weights to responses - Collect mean, std dev, median Semantic differential - semi graphical method - select antonyms for polar anchors - Loved it __:__:__:__:__:__:__ Hated it - Can assign some weights to responses - Collect mean, std dev, median Numerical Scaling - uses bipolar adjectives like semantic differential but uses numbers instead of blanks - may include one internal middle of the road - How would you rate the promptness of the server? Very Slow Acceptable Very Fast 1 2 3 4 5 - Odd number of choices allows indifference - Even number forces a choice - Can assign some weights to responses - Collect mean, std dev, median Graphic Scales - Provide a continuous (vs discrete) scale for response - How would you rate the cleanliness of the server? Very Clean_________________________________Very Dirty - Must measure with a ruler - Can assign some weights to responses - Collect mean, std dev, median Paired Comparison direct question of preferring A vs B n(n-1)/2 questions Sorting good for placing a large number of choices into relatively few categories e.g. Here is a stack of cards containing the names of 100 manufacturing companies. Go through the pile and sort it into those you've heard of and those you haven't. Now take the stack of the "heard of" and sort it into those whose products you own and those whose products you don't know. Essentially count the number of times any item appears in a category Rankings Give a list of related choices, ask respondent for a preferential ordering Can use a total of all rankings, number of times appearing in a specific position ( how many ranked this #1) Constant Sum Give respondent a bundle of something to allocate among a finite number of choices Keep choices to a few or people find it too complicated WRITING QUESTIONS Ideally a question should : * Perfectly understood by all respondents * Completely objective -- no influence * Allow for only one interpretation * Aid in answering the research question * Elicit a truthful response * Create no conflict in the respondent PITFALLS Leading Questions -- suggests a response - Do you believe that people over 50 can be just as productive as those under 50? - Would you buy this fine product? Loaded Questions -- Emotionally charged or suggestive - Do you agree that abortion is murder? - How should the law punish habitual abusers of drugs? - Good housekeepers will wax their floor at least once a week. How often do you wax the floor? - Savings and Loan cheats, like Charles Keating, should be punished with jail time. Sensitive issues -- embarrassment, anger, prone to lie - Salary - Education - Sexual activities/preferences - Religious fervor/preferences/activity level Asking more than one question -- lawyer trick - Have you stopped beating your spouse? - Have you spent an excessive amount of money on food and entertainment? Asking vague questions -- too global - Do you approve of the way the government handled the S&L crisis? - How important is color in your selection of a car? Making assumptions -- How bad/good before knowing it exists? - What is your biggest problem with the police department? - How would rate your college education? - Using big words - K.I.S.S. more people understand "walk" than "ambulate" - Must walk the line between insulting the intelligence of the respondent and making him/her feel ignorant Poor ordering - Order bias -- respondents will tend to answer similar questions in a similar way - Mix up order on scaled questions -- flip "positive" and "negative" poles - Mix up types of questions--open and closed, scaled, comparison - Lead with "interest grabber" end with sensitive DATA GRUBBING MUCH CAN BE LEARNED FROM EXPLORING HISTORICAL INFORMATION STATISTICS IS THE ONLY "AFTER THE FACT" SCIENTIFIC METHOD AND MAKES IT POSSIBLE TO USE HISTORICAL DATA EXTERNAL -- public domain, previous research Try to find everything that has been done or is relevant! BIG JOB! Agonizing procedure, no cookbook Where to begin? - defined by research question - identifying keywords in question + synonyms - identifying formal names (company's, authors, agencies, programs) - identifying key periods (dates) Using a database search -- SQL - .AND. -- make smaller - .OR. -- make bigger - .NOT. -- negate - precedence order of calculations What will you get? - usually a huge list (too big) - use .AND. to narrow - dump a reasonable list Usually read a few articles - check bibliography - backward chain until happy TIPS - use the librarians they give PHDs in library science - make best use of your time (when in doubt XEROX) - don't try to get too much the first day - most articles provide a couple paragraphs and some references - check to see that you have bibliographic reference INTERNAL -- Part of organization, program, history Not as easy as it looks - information is power - has extrinsic value - may be disguised - may be hard to find Must know Who: - pays - uses - authorizes - provides information - loses / wins If not the same you got trouble Knowledge as the true value in organization - depreciable -- has time value - political -- "I know something you don't know" - well guarded -- "Why do you want know?" - well hidden -- a straight answer from an accountant!?! Often will require the use of "proxy" variables - can't always get what you want - may have to find something that is highly correlated - e.g. Length of Stay => Severity of Illness? - e.g. Rank/Title => Income Must be aware of - anomalous occurrences e.g. Exxon Valdez, take overs, lawsuits - those with institutional history => interpretations - effects of perspective => to marketer, all solutions are marketing - settle for averages e.g. $/truck = Cost tires + Cost parts +.../#Trucks - transformation requirements annual => unit cost - merging from different sources will leave holes Characters you will meet: - "data cookers" -- paranoids - "obstructionists" -- resistance to change - "ostriches" -- don't really want to know - "envious" -- jealous of your initiative - "saboteurs" -- anything they can to set you up - "irrational" -- I'm against it - "druids" -- gate keepers and guardians if you can possibly avoid it: - never go through channels -- grunt to grunt - never ask for commitments on paper - never volunteer more than you have to - never ask for permission, wait till you get your hand slapped - never promise as much as you really think you can deliver SCALING AND MEASUREMENT THIS IS NOT A TRIVIAL ISSUE! HOW YOU MEASURE DETERMINES WHAT YOU CAN SAY ABOUT RESULTS TYPES OF SCALES NOMINAL -- Labels only - simplest type - limited analysis potential -- mode, frequency distribution ORDINAL -- Order but no distance -- ranking - typically the "strong agree --> "strong disagree" - median, percentiles are best potential measures CONTINUOUS -- Order and relative distance - absolute values ("place X on the line") - mean, standard deviation, z scores - can use integer values SOME CRITERIA FOR JUDGING SCALES RELIABILITY -- If I do it again will I get the same answer? May be problem with subjects, techniques or extraneous variables Repeatability -- if I ask again will I get the same results - things change over time e.g. boundary with a river bed - inherent variability in subject pool (moods, recency) Internal Consistency -- are subjects taking it seriously - same question, inverted sense A to B => B to A - split answers compare halves for correlation - restated questions -- measure the same thing, different phrases ACCURACY -- How close to the truth is it? - does it actually measure what you want? (income => success?) - does respondent agree with question - does it address my hypotheses - is the scale real PRECISION -- How tight is the span of truth? - may be accurate but imprecise (balloon story) - close enough - do we use a bucket for measuring vanilla? - do we use an eyedropper to measure swimming pools - significant figures -- measure in lbs => don't report grams OTHER CONCEPTS YOU MAY FIND USEFUL HIDE SENSITIVE DATA, OR COMPOSITE MORE THAN ONE VARIABLE INDEXING -- Pick a base, divide everything by the base - wholesale prices - can combine multiple factors into a composite measure(TOTSCORE) - way to hide absolute values DEVIATIONS - how far from "average" - another way to hide info - descriptive value RATIOS - divide one measure by another - implied correlation - deal with order of magnitude changes - good for "unreachable" data (billions) - e.g. "constant" vs "actual" dollars SAMPLING WHY SAMPLE? ~ Can't poll the whole "population" (All students) ~ May "destroy" the utility of the sample (light bulbs) ~ Can't reach the whole "population" ~ Can usually get close enough NON RANDOM SAMPLES Convenience -- who ever is handy - may not be representative - can't really conclude anything - cheap Judgment -- Researcher decides which subjects are representative - really quick - subject selection not critical (Consumer Price Index) - good for longitudinal work - may rely on history being a predictor of future Quota Samples -- 10 apples, 8 oranges, 5 bananas - Ok if subgroups are accurate - Not random - Can be biased by convenience Snowball Samples -- Find a rare one, ask for more rare ones - good for exploratory definition of rare attributes - bias may be introduced (similar subjects) - may miss some with attributes who are not similar - good ol boy network RANDOM SAMPLING * VERY SCIENTIFIC * VERY POWERFUL -- IF IT'S DONE RIGHT (STATS REVIEW) * NOT AS SIMPLE AS IS THOUGHT * NOT THE SAME AS UNSTRUCTURED Typically we use a random number table as the basis for choice TYPES OF RANDOM SAMPLES SIMPLE -- Everybody has an equal chance "names in a drum" - most powerful for generalizing - careful selection of population required SYSTEMATIC -- Every nth candidate from an ordered list - systematic sample of a random list is a simple random sample - recognize distribution of list (periodicity) BLOCKED OR STRATIFIED -- Random choice from a set of subgroups - usually breakdown a population on the basis of expected differences between groups - may be weighted by some factor - Nielsen TV ratings use this method - If done right, much stronger than random by ensuring accurate representation - Example -- Suppose you have a strong reason to believe that rank will be a strong factor in response. To ensure accurate representation of your sample you could: Choose a suitable number of categories: - commissioned - NCO - enlisted - Find the proportion of each category on the base (e.g.) - 15% commissioned - 25% NCO - 60% enlisted - Select a sample size (say 200) - Randomly choose - 30 commissioned - 50 NCO - 120 enlisted Cluster Sampling -- Random within a random group - 200 banks could be done in 4 randomly selected cities - assumes all cities will have the same distribution in each - usually geographical groupings HOW BIG A SAMPLE? NEED TO REVIEW STATISTICS TO ANSWER EXPERIMENTAL RESEARCH EXPERIMENTS ARE CONTRIVED SITUATIONS THAT ALLOW THE RESEARCHER TO MAXIMIZE CONTROL OVER IRRELEVANT FACTORS AND TO OBSERVE AND MEASURE THE EFFECTS OF MANIPULATING ONE OR MORE INDEPENDENT VARIABLES ON A DEPENDENT VARIABLE. UPSIDE + isolate the effects of the study variables + minimize effects of confounding factors + "prove" causal relationships + tough to argue with results DOWNSIDE - hard to eliminate extraneous influence - hard to design well - usually expensive - external validity questions - tough to explain results TERMS SUBJECT An individual or physical entity that will be manipulated by the experiment. (may be a person or a petri dish) TREATMENT One manifestation of an independent variable (e.g. "Frequency", "Duration") LEVEL A measured state of a treatment. (e.g. Hi, Lo, 100 degrees) CELL A group of subjects who experience a unique combination of treatment levels CONTROL A treatment of the experiment that represents the "normal" state for the subjects. The control group will receive no manipulation from the experiment. CHOOSING VARIABLES Should be something you can easily observe or manipulate in the real world Should not be measures of similar things (e.g. income and house value should be highly correlated) Should consider measurement scales and methods Should answer the research question RANDOMIZATION--MOST POWERFUL VALIDATING TOOL AVAILABLE ~ Try to make sure that there is no bias among cells. ~ Only assures equal opportunity for assignment ~ COMPLETELY RANDOM DESIGN - most powerful - random assignment of each subject to a cell - randomly distributes all extraneous conditions among cells - Example: For my experiment there were 10 cells. As each subject arrived, he/she randomly chose a card from my hand with one of the ten treatments. ~ BLOCKED DESIGN - expectation that some pre-existing attribute of subject will cause a difference in performance - still uses a random assignment of subjects who are "blocked" by attributes An example of a simple blocked experiment You are attempting to determine which of two films (A or B) will be more effective in heightening consumer awareness of breast cancer. You will measure the change by administering a questionnaire both before and after showing the films. You have a subject pool of 24 women and 18 men. Some alternative designs: Randomly assign each of the 42 subjects to one of two treatments. - most risky - least data - most powerful conclusions Randomly assign each of the 18 men to treatment A or B, randomly assign each of the 24 women to treatment A or B. - more data (by sex) - still only checks for A vs B - does not control for learning effects on questionnaire Randomly assign each of the 42 subjects to A,B,Control (Control will watch an equivalent film on an unrelated subject. - controls for any questionnaire bias - if significant more powerful, internal validity - lose some advantage of numbers to control Randomly assign 6 men and 8 women to A,B or control. - best overall ( if women and men are different) - if not blocked design does not invalidate results Extraneous Variables -- Can something not controlled by the experimenter explain the results. (Are all cells equivalent except for treatments) Demand Effects (Hawthorne Effect) -- Does knowing about the experiment alter the subjects reaction Learning Effects -- If a repeated or sequential series of actions occurs, do subjects acquire skill as they go along Environmental Bias -- Do the conditions (or the experimenter) change with individual treatments or cells. Maybe a result of time Time Effects -- Are conditions the same at the end of the experiment as they are at the beginning. Mortality -- In longitudinal studies, subjects may disappear Conspiracy -- Do subjects interact socially or during the experiment Motivation -- Why should the subjects care? SOME MITIGATING MEASURES ~ Sequence Balancing -- if there are two treatments, some do A first, the rest do B first. Other variants: - Latin Square A B C, each treatment appears in only one B C A column and one row ; equal number of C A B factors and treatments - Inverted control XTX, TXT ~ Blinding -- Subjects or experimenter (or both) are kept from knowing what treatment will be performed ~ Covariate -- include an independent measure to test for presence of suspected bias FACTORIAL DESIGNS ~ very powerful (get more than one subject per treatment) ~ main effects ~ interaction effects ~ require approximate balance A 2 X 2 Factorial Design A bookstore chain wants to know which of two cover colors will be most effective in promoting a new author's book. In NY and LA three store locations are chosen for the experiment. Books are placed on a display with either a RED or BLACK cover. Each book has a different barcode so that sales can be tracked by color. All colors are present on each display. Sales for a week will be tracked. RED BLACK NY SALES(1,1) SALES(1,2) Cost NY SALES LA SALES(2,1) SALES(2,2) Cost LA SALES Cost RED SALES Cost BLACK SALES WHAT INFORMATION DO WE GET? (Dependent Variables) - Total Sales by Color (Main) - Total Sales by Store (Main) - Sales by Color and Store (Interaction) WHAT CONTROLS SHOULD BE IMPLEMENTED? - maintain inventory levels of all colors - same time period(s) - blind store employees - same location in store - same price - same store hours WHAT ELSE MIGHT WE CONSIDER RELEVANT? - time periods (per week) (3rd Dimension) - % of sales for this volume (alternative dependent) - total sales for store (covariate) WE WILL ANALYZE THIS MODEL IN THE NEXT SECTION REPORTS AND PRESENTATIONS THIS IS YOUR PRODUCT! ~ Its a drag to do in most cases ~ May consume 20-30% of your total time ~ Poor job on report => questionable results ~ Really a sales job DIFFICULTIES ~ Research skills do not imply communications skills ~ Audience - "I don't know what I want but I know THAT isn't it" ~ You are too close to the work ~ You never know where it will wind up ~ You may be have to work with a diverse audience GOAL - TO CONVEY THE ESSENTIAL INFORMATION AS SUCCINCTLY AS POSSIBLE Hierarchic Organization -- "If you want to know more, read on" Typical Organization (One way,not the only way) Executive Summary or Abstract - 1 page - key conclusions/recommendations - toughest to write - what action should be taken if any Organization Overhead - House keeping - Transmittal letter - "It's done" - Authorization letter - Who paid - Table of Contents (List of Figures, Tables) Report Summary - The nutshell version - Objectives - Results - Conclusions - Recommendations Body of Report - "The story" - Background - How we got into it - Objectives - What we set out to do - Methodology - How we tried to do it - Results - What we found - Critique of results - What we did right and wrong - Conclusions - Answers to research question - Recommendations - Implications of answers - Direction for further work - What is the next step Appendices - Details for those who want them - Forms, scripts, questionnaire blanks - Detailed calculations, mathematical models - Tables of summary data - Other supporting material - Bibliography references ESSENTIALS OF WRITTEN BUSINESS COMMUNICATION ~ Audience Recognition - Don't over estimate sophistication (too many acronyms) - Don't patronize - How much do they know ~ Writing style - Use simple declarative sentences where possible - Don't get overly abstract (analogies) - Paragraphs deal with one idea - Avoid passive tense - Use "spoken" style (would you verbalize this way) - Maintain tone and tense - use bullet lists, outlines where appropriate ~ Directing/leading the reader - transitions should be clear and smooth - framing "we are doing this to answer..." - logical evolution of ideas ~ Organization - modern 1 CHAPTER (always a new page) 1.1 SECTION 1.1.1 Sub Section 1.1.1.1 Paragraph ORAL PRESENTATIONS IN CONTEMPORARY BUSINESSES THIS IS AN IMPORTANT PROCESS WHY? ~ Time -- Reach a lot of folks at same time ~ Efficiency -- Answer the same questions for everybody ~ Consistency of Understanding -- Everybody on same footing ~ Interactive -- Immediate feed back ~ Group Synergy -- Interchange of ideas ~ Credibility -- People will size you up, decide whether you should be believed (proliferation of information) From Researcher's Perspective: + control agenda + persuasive capability + nuance of spoken word - can be pushed off on tangents - swept up in politics - poor job can destroy credibility GUIDELINES FOR ORAL PRESENTATIONS Primacy and Recency -- People will remember the first thing you tell them and the last thing you tell them General Process ( Repeat as necessary) - Preview -- tell them what you will tell them - Body -- tell them - Summarize -- tell them what you told them Visual Aids - Keep them simple 3 to 7 pieces of information - Don't read them to audience - Use sparingly to reenforce [1 per 1 to 5 minutes] - Make sure they are well organized Handouts - use very sparingly (audience should listen not read) - NEVER handout your report prior to presentation - one page outline may help keep people focused DO - have an outline - make eye contact with audience - watch your time DO NOT - write a script - read from your notes - lose your sense of humor Handling Common Problems Controlling the Agenda - post an outline before hand - stick to it - tell audience how you wish to handle questions before hand - don't be afraid to defer answers to a more appropriate time - don't let audience members argue with each other Your own anxiety - confess, most people will empathize - select one person to talk to (rotate) - hold a prop Antagonists -- You MUST deal with them usually at back or sides of room with "angry" body language (crossed arms, disgusted expression) always agree with criticisms -- very disarming "That doesn't wash, you should have..." "I probably should have, but I didn't ... Never get riled, use the audience to control Try walking to a point where the bulk of the audience is between you and the antagonist. This will force the rest of the audience to either focus on you or to assume an uncomfortable posture to see the antagonist don't argue, try to defer question to a later time or suggest that others have questions and you will get back to him/her VISUAL MEDIA OVERHEADS + do in advance + can highlight if necessary + easy to do (xerox) + easy to copy - projectors are bulky - limited to portrait orientation - must color by hand - must be posted by hand - messy to sequence - lights off SLIDES + use whatever colors, exotica you wish + compact + pre sequenced + you can move around - expensive - requires outside services - limited format - no highlighting - lights off FLIP CHARTS + easy to do + inexpensive + personal + very portable + lights on + interactive - must be done by hand - small groups - can't erase highlights - can't copy BLACKBOARDS /WHITE BOARDS + interactive + no read ahead + easily adapted to audience + selective erasure + lights on - not very portable - no permanent record - no copies - requires presenter to write GRAPHS AND CHARTS Pie Charts - percentage of the whole Stacked Bar - multiple pies - contrast changes in percentages or absolute levels Area or Stratum - stacked bar for continuous variables Line, XY charts - changes over time - continuous variables - best when levels change/cross Bar Charts - contrast levels for discrete variables ETHICAL ISSUES IN RESEARCH DECEPTION - sometimes used in experiments - often attempted by not telling subjects the whole truth - if done, you must debrief RIGHT TO KNOW - it is both illegal and unethical to misrepresent your purpose - if requested, you are obliged to disclose your purpose PRIVACY AND CONFIDENTIALITY Be aware of the subjects right to privacy. There are regulations concerning disclosure of things like SSN, ethnicity, religion, and financial information. Despite the proliferation of electronic databases, you must be careful about requesting or disclosing this kind of information. Info you gather is between you and your client/supervisor. You must protect this relationship to maintain any long term credibility. You may have to cut a deal with sources to obtain information. Is your client then entitled to the same specific info? Could client legitimately obtain this info without you? We talked about ways to report and aggregate data to protect confidentiality. When in doubt, use them. There is a large temptation to "reuse" and "repackage" data from prior studies -- resist COOKING DATA AND REPORTING RESULTS A legitimate function in research is to identify and discard "outliers" in data sets. These may be observations that you know are not representative. Often the difference between significant and non-significant results will be one or two observations. Are these "outliers"? If you don't find cancer, will you get another grant? THE GENTLE ART OF PERSUASION -- Where to draw the distinction between reporting and sales. One of your objectives is to build a convincing argument for your conclusions and to defend them against attack. Good research raises more questions than it answers You have an obligation to critique your results, and suggest the direction for future work. How far do your results really reach?