Saturday, September 13, 2014
Thursday, September 11, 2014
Monday, September 8, 2014
Structured role-playing is a method that forecasts an individual’s or organization’s preferred response to a given problem by simulating both problems and interactions between protagonists using interactive role players in a scenario designed to estimate the intentions of protagonists who may have disparate interests. The interactions between the role players serve as a proxy for the interactions of the decision makers and the decision reached by the role player is taken as a forecast of the decision maker’s preferred response.
1. It may be less expensive than other methods
2. Provides additional outcomes not considered by experts on opposing objectives and strategies
3. Easy exercise to explain
4. Each exercise can be structured to fit any scenario
5. Can use a large pool of role-playing participants
6. The role-playing participants are less likely to fall into “group think”
1. Lack of preparation for roles negatively impacts quality of the exercise
2. Repeating exact results will likely be difficult
3. Falling victim to stereotypes if roles are not constructed adequately
4. No agreed upon criteria for evaluating role-playing exercises
5. Finding students with similar backgrounds may be difficult to find
Step by Step Action:
The first step involves participants becoming familiarized with the roles they will be playing. The participants need time to dive into their characters and start to think about how that person acts and feels. This goes beyond what is provided within the description. The participant should think about how that person lives, what do they do during the weekend, who their friends are, etc..
Now that participants are familiar with their roles, they should be briefed on the situation or the question that needs to be answered. The participants will apply what their understanding of their role is to the scenario and develop a response to the situation.
Once the simulation has ended, the participants will record how they feel their character would act in the scenario. The participants should record not only what they believe the outcome will be, but also their thought process. The thought process will provide valuable evidence into how the participants viewed their characters and potential areas for further research.
Participants were given 1 of 2 roles. One role was Marcus, a 22-year old African American male from a socially-ill neighborhood in Erie. Marcus has been in and out of the criminal justice system as a juvenile and continued his misbehavior into his adulthood. As a result, Marcus is currently on probation. The other role was Margaret, a middle-aged suburban mother of two college graduates. She works part-time as a bookkeeper and spends the remainder of her time either gardening or volunteering at the homeless shelter.
After familiarizing themselves with 1 of 2 roles, the administrator gave the exact same scenario to each participant. In the scenario, a large male approaches the participant (Marcus or Margaret), asks to use a cellphone, then assaults the participant and steals the phone. After the assault, the participant is taken to a nearby hospital, where a nurse notices injuries of the assault and offers to call the police on their behalf. Participants are asked whether their character would call the police.
While the sample size was extremely small, participants playing Marcus opted to not call the call police. Participants cited Marcus’s probation and past criminal inclinations for reasons to not report the incident. Conversely, those playing Margaret decided they would call the police. Participant playing Margaret cited her fairly routine and risk-averse lifestyle as reasons why Margaret would report the incident.
Friday, September 5, 2014
This article published in 2011 by Kesten Green, Senior Lecturer at Ehrenberg-Bass Institute for Marketing Science at the University of South Australia and Scott Armstrong, Professor of Marketing at The Wharton School at the University of Pennsylvania highlights the significant differences between simulated interactions and a term they coined "role thinking" on the accuracy of decision forecasts in novel situations. They used an experimental design to conclude that asking groups of people to think about the roles and interactions influencing the reactions of a protagonist in a given novel conflict situation to forecast protagonist decisions is an ineffective forecasting technique. Forecasts from role thinking are unlikely to be accurate due to the difficulty of analyzing complex interactions between different protagonists with different roles in a manner that accurately represents the conflict in the absence of experiencing the complex interactions.
Note that the authors prefer the term "simulated interaction" instead of "role playing" to refer to the method of forecasting people's decisions by simulating the situation using interacting role players because the term "role playing" is used to refer to various techniques with purposes other than forecasting. The authors find that simulated interactions provide much better forecasting accuracy than unaided judgment and role thinking, particularly in novel conflict situations.
Evidence from previous findings indicate that much better forecasting accuracy for the decisions the protagonist in a given novel conflict situation will make is attained by prompting groups of people to adopt roles in addition to simulating the interactions between protagonist groups with divergent interests. The decision the protagonists in each group elect to make in the simulated interaction is taken as a forecast of the actual protagonist's decision. The authors highlight that simulating novel conflict situations faced by divergent protagonists using interacting role playing members solves the absence of experience problem according to prior research from the same authors using the same conflicts utilized in this experiment.
The authors tested role thinking in an experimental design consisting of forecasts for the decisions of the protagonist of a given conflict from an expert sample and a novice sample. The accuracy of the role thinking forecasts were compared to chance in addition to the accuracy of unaided judgment and simulated interactions from previous studies utilizing the same conflict scenarios. The authors obtained 101 role thinking forecasts for nine conflicts from 27 Naval postgraduate students (the expert sample) and 107 role thinking forecasts from 103 second year organizational behavior students (the novice sample). The results are illustrated below:
The average forecasting accuracy from the novice sample and the expert sample were only marginally better than chance, which was 28% versus the 33% accuracy from the novice forecasts and the 31% accuracy from the expert forecasts. Previous research from Green and Armstrong using the same conflict situations found forecasting accuracy of 60% when using the simulated interaction method instead of the role thinking method.
In the role thinking experiments, participants were provided with descriptions of some or all of the situations and of all the associated roles. Participants were prompted to predict what actions each party in the situation would prefer and assess how likely it is that each party's preferred decision will actually occur. Each prompt had a list of between three and six decisions that the researchers believed could plausibly have been made in each situation.
In previous simulated interaction experiments with forecasting accuracy of 60%, participants were divided into groups and assigned information only on their own role. Participants were prompted to read their role description, put on a name badge for the role, and adopt the role for the duration of the simulation. Participants were free to meet with others as often as they would like to reach a decision. Each group's decision was taken as a forecast of the actual protagonist's decision. In addition to the 60% accuracy finding, the forecasts from simulated interactions were more accurate than the role thinking forecasts for all nine conflict situations. The authors also point out that neither statistical nor casual models have been found to be feasible for predicting decisions people make in novel conflict situations therefore decision makers rely on judgmental methods.
The forecasts from the role thinking experiment were derived from individuals while the forecasts from the simulated interaction experiments were derived from group forecasts. The authors acknowledge that a key assumption driving their analysis of the ineffectiveness of role thinking versus simulated interactions is that forecasting accuracy from group role thinking forecasts would differ little from the individual role thinking forecasts in this experiment due to both unaided judgment and role thinking forecasts differing little from chance. One way to test this assumption is implementing an experiment where different groups of participants arrive at a group forecast using role thinking then comparing those results to the simulated interaction experiments. This would also be a more consistent representation of what occurs when a group of people are tasked to engage in role thinking together on a team.
In Table 2, the expert sample in the unaided judgment experiments and the expert sample in the role thinking experiment are qualitatively different in that the unaided judgment expert sample participants were from academia and professional conflict management and forecasting organizations while the expert sample participants in the role thinking experiment were Naval postgraduate students. However, the authors point out that there is little evidence that top experts can perform judgmental tasks better than generalists. In addition, the naval postgraduates had experience in conflicts over pay negotiations and commercial takeovers, the authors suggest that knowledge of conflicts from one domain is likely to transfer over to other domains involving predicting human behavior in conflict situations.
One thing I did not see addressed is best practice for developing effective scenarios, roles, and choices for simulated interactions when the conflict being assessed is a current event. I estimate that the procedures for developing effective simulated interactions is as much an art as it is a science, especially when the simulated scenario in question is a model of a current novel situation where incomplete or intentionally deceptive information is an issue. Using simulated interactions with current conflicts and events as opposed to historical situations where the outcome is known by the developers of the simulated interaction in advance adds a layer of complexity to developing the simulation interaction requiring further study.
Green, K. and Armstrong, J.S. (2011). Role Thinking: Standing in Other People's Shoes to Forecast Decisions in Conflicts. International Journal of Forecasting. Vol. 27(1). p. 69-80.
Forecasting decisions in conflict situations: a comparison of game theory, role-playing, and unaided judgement
SummaryThis article presents a study done by Belova, N., Feierabend, T., and Eilks, I. to evaluate role playing in science-based scenarios. Belova et al. stipulates that although many applications for role playing in science education have been developed, there is limited research on the performance during the exercises. Additionally, Belova et al. points out that there is no agreed upon method for analyzing arguments within a role playing exercise. This paper presents a systematic method for analyzing performances in role plays and what level of argumentation and decision-making capabilities are evident within the exercises.
Belova et al. performed the study by having 20 groups of
students, aged 15 to 17, participate in the role playing exercise of debating
climate change as a stake holder, such as the automobile industry or Greenpeace
activists. The 20 groups were assigned to one of four class subjects, biology,
physics, chemistry, or politics which then became their primary source of
knowledge for the exercise. The exercises were recorded and evaluated by two
By: J. Scott Armstrong
This publication outlines the basic elements of role-playing using a series of studies. The author conducted a study that had two groups of two participants each for 80-minute sessions. Each person handled one simulation as an expert and another as a role player. In each of the simulations, each person received closed ended questions ranging in possible decision making conclusions. In each of the role-playing simulations, participants were assigned background information to make the simulation sound realistic while viewing the problem from an outside perspective.
After reading the material for 20 minutes, each party would meet at a table until they reached a consensus or when the time ran out. After the role-playing was completed, each person answered a set of questions. These questions instructed them to state the consensus as they saw it or, if they did not reach a consensus, what would have happened if they reached a conclusion in the appropriate time.
Also, to reduce the chance of misinterpretation, participants were asked to write their views of the decision down independently at the end of the simulation. This is a great exercise to do since it forces people to agree on the underlying decision.
Some of the advantages of role-playing are that it is less expensive than experiments and it provides additional outcomes not considered by experts on opposing objectives and strategies. It provides individuals to think for themselves. Although the author supports role-playing, this publication has some disadvantages.
One of the basic elements of role-playing is to cast roles similar to the people they represent. According to the author, participants should have similar backgrounds, attitudes and objectives to the role-playing simulation. In addition, Ashton and Krammer (1980), “found considerable similarities between students and non-students in studies on decisions making processes.” While valuable, it does not state whether undergraduates partook. If so, how can an undergraduate student have a similar background and attitude towards a role-playing situation if s/he does not have a lot of real work experience and/or an education in the topic of discussion? Therefore, further research needs to show a more in depth correlation stating why students and non-student are showing similar role-playing conclusions.
In addition, the author stated that each role-playing simulation “should run around ten sessions, five using one description and five using another.” The author did not go into detail on why he chose ten sessions versus any other number. Are ten sessions the average used in role-playing simulations, or is that the number he likes to use?
Armstrong, J. S. 2002. Principles of Forecasting: A Handbook for Researchers and Practitioners. 2001 edition. Boston, MA: Springer.
Wednesday, September 3, 2014
Thursday, May 9, 2013
Meditation is an analytic modifier that allows for the training of the mind by promoting relaxation. The outcome of this practice may lead to improving health issues including high blood pressure, anxiety, and depression. There are a wide variety of different meditation techniques; examples include basic meditation, focused meditation, activity-oriented meditation, mindfulness meditation and spiritual meditation. These activities are intended to reduce stress, improve multitasking, processing speed, attention span, coordination, and cognitive flexibility.
1. Some studies show that it has potential to decrease stress, improve mood, and improve health.
2. Improves the practitioner’s ability to control his or her own thoughts.
3. Studies have demonstrated meditation’s potential to improve multitasking, processing speed, attention span, coordination, and cognitive flexibility.
1. May initially cause more stress.
2. May need long-term training to see significant results.
3. Meditation is relatively non conventional and therefore requires an open mind.
4. Individual needs to practice meditation on a daily basis to maintains results.
Step by Step Action:
1. Choose a type of meditation which you find to be best for you. These can include basic, mindfulness, or spiritual meditation. The following link lists additional types of meditation and their descriptions: http://stress.about.com/od/lowstresslifestyle/a/meditation.htm
2. Set aside 15 minutes each day to conduct this exercise.
3. If you are trying basic meditation, follow the subsequent steps:
- Sit in a comfortable position and quiet your mind.
- Notice the thoughts in your mind, but do not engage them. When thoughts materialize, just let them go.
4. Continue practicing each day and with time, it will become more natural and less difficult.
Before coming into class we were told to set aside 15 minutes to engage in one of five meditation techniques. These five meditation techniques included: basic meditation, focused meditation, activity-oriented meditation, mindfulness technique, and spiritual meditation. Each participant reflected on how they felt before and after conducting the 15 minute meditation session. The most popular meditation technique that was chosen in the class was basic meditation technique, with focused meditation technique as another popular choice. One problem that was noticed that many participants prior to conducting their meditation were significantly stressed out before and had a difficult time at first getting stressors out of their minds. However, the longer each participant practiced the technique the easier it was to clear ones minds of what was stressing them out.
As a class we discussed the usefulness of meditation and its utility to be instituted in the intelligence community. Through the class discussions it was discussed that meditation was useful in improving multitasking, improved processing speed and cognitive flexibility. Overall, it was noted that meditation techniques would be useful to institute within the intelligence community, but the ways in which to best institute them still needs to be clarified. There are multiple meditation techniques that have demonstrated the ability to be beneficial for the participant, but there is not just one definitive meditation technique that is proven to demonstrate the most positive results. If implemented into the intelligence community it would be necessary to expose individuals to a variety of different meditation techniques to find which one would work the most effectively for them.
- Can reduce stress, improve multitasking, and improve concentration
- Can creative mindfulness that results in general improved positivity towards life
- Likely requires repetition and frequent implementation to become effective
- Requires practitioners to be receptive of the approach
- Difficulties with quantifying any benefits (ie: how effective meditation can be)
- Research different techniques of meditation and select the one you feel is most comfortable (ex.: basic meditation, activity-oriented meditation, etc.)
- Find a place where you are comfortable to meditate
- Carry out your chosen meditative technique