Research methodologies

The general structure of master thesis work was presented as this:

• Determine topic: Something that interest you and that falls within your study program
  
• Review the literature: Goal is to narrow down your interests, figure out what methods and approaches has already been tried out, get inspired, look for common ways of measuring variables thought to affect the area of interest and what results to expect in general. Like how reliable current methods are.
  
• Find questions: Find specific problem that are focused, but not too focused. This is performed together with your supervisor.
  
• Create hypothesis: Your hypothesis is proposed solutions or explanation to the research problems that you want to test using some method. Looking at expected outcomes is important and the hypothesis must be testable.
  
• Operationalize: Figure out how to measure variables related to your hypothesis
  
• Experiment: The most common are surveys, experiments and field research. Surveys are common in social sciences, experiment primarily means measuring other things inside a laboratory or model. Field research is describing the real world outside of the laboratory and is more in the direction of qualitative research.
  
• Analyze: Use statistics and explain why you got the result you got, independent of being expected or not.
  

Very important goals of any research is to make it unbiased meaning we only want to measure what really is. This is often very difficult. Typically we need to test on samples or models of the world and thus might end up with results that are not representable.

Typical errors:

• Selective observations: Like "selective hearing", find evidence support one side. Performing literature study will help finding alternative ways to look at a situation. Also looking for dis-confirming evidence is a strategy for avoiding this.
  
• Inaccurate observations: Failure to record "measurements" fully or using the "instruments" in a wrong way. Note that a measurement can be an answer to a question and instrument can be interview.
  
• Over-generalization: Apply a concept tested on a small sample to the whole population. Good research questions can be to test previous research in new contexts with new assumptions.
  
• Made up information: The worst case is fabrication of supporting evidence. Milder versions include guessing for missing observations. Reconstruction methods can be applied (they are basically assumptions)
  
• Ex post facto hypothesizing: "After the fact", When testing a hypothesis, one must never look at the results and then alter the hypothesis to support the evidence but instead come up with a new hypothesis, predict new outcomes and test them using a new experiment different from the original. This is to avoid mixing correlation with causation. An experiment is supposed to control for variables so that causation can be found and this does not work in reverse.
  
• Illogical reasoning: Like basing the research on wrong premises or making logical mistakes. Best way to avoid is peer review.
  
• Ego involvement: Promoting ones own work with higher regard than others, the importance of it and how much is contributed to one self.
  
• Premature closure of inquiry: Research is an iterative process but at some point one must stop of practical reasons. Premature stopping, like an experiment is often a huge source of why the "correct" answers are not found. Still, be honest, speculate in possible shortcoming of stopping now and don't mystify hard problems. They can still be solved by others.
  

This lecture were very focused on quantitative research and did not cover much of qualitative approaches. We had a lecture on typical methods for computer science and engineering during scientific methodology class that might help in understanding how our problems in information security (not involving algorithms) can be approached.