There are many unknowns in the world. This holds true for those on the fringes of scientific inquiry, business, and even our personal lives.
Oftentimes, there’s no way to know the answers to questions with 100% certainty so what do we do? We test out different things and figure out which one works. This is known formally as experimental research and you’ve likely done it many times in your life whether you realized it or not.
In this guide, we’ll take a deep dive into experimental research so you’ll know what it is, how to do it properly, and when it should be used.
Definition of experimental research
Experimental research can be defined as a formal or scientific approach to research. Researchers manipulate one more variables with one of them acting as the control while the other variables are changed to gauge the impact those changes have on the subject of the experiment.
For example, you have two stalks of grass. The control group is planted and kept where there’s sunlight. One of the experimental groups is kept in a dark cupboard and another one has its roots submerged in water.
Many people will be familiar with this from middle school and high school experiments. Even the most advanced research studies today still take advantage of experimental research.
At its heart, experimental research is a type of trial and error that allows you to test hypotheses or improve over time.
Types of experimental research design
Experimental design comes in many forms. The one you choose will depend on various factors such as your expertise/resources, the aim of the experiment, the acceptable margin of error, etc. Let’s look at the most prevalent forms of experimental research design.
True experimental research design
In this type of research design, the results aren’t just observed, statistical analysis is used to determine if the changes noted are significant. Because of this, it’s considered the most accurate form of experimental research design.
Before research can be considered true experimental research, it first needs to fulfill three criteria.
- There are two or more groups and one of the groups is a control which won’t experience any deliberate changes throughout the experiment
- There is a variable that can be changed by the people carrying out the research
- There’s a random sampling as opposed to a curated sample for the groups/subjects
Pre-experimental research design
This is the most common type of experimental design and also the simplest to achieve. One or more groups or subjects are placed under observation after factors that are believed to cause change are introduced. There is no control group and the goal of this experiment is to understand whether or not the variable you’re interested in is worth spending more time researching.
There are multiple types of pre-experimental research design:
- One-shot case study – One variable is considered and it’s analyzed after the group has been exposed or treated with the factor that is believed to elicit change.
- One-group pretest-posttest – Analysis is carried out on a single group before the factor believed to cause change is introduced and after it has been introduced.
- Static group comparison – There are multiple groups – usually two – and one is used as a control while the other one is exposed to the factor that’s believed to cause change. After introduction, both the control group and the experimental group are examined. Any changes/differences noted between the groups are credited to the factor that was introduced.
Quasi-experimental research design
Quasi experimental research design is closely related to true experimental research design. It has a control group but the difference is that the subjects aren’t randomly assigned like they are in a true experimental research study.
This type of research is done when randomly assigning a control group is difficult or irrelevant to the end result. Apart from that, the researcher still manipulates a variable and observes the changes that occur.
Experimental research use cases
Like many other types of research, there are countless ways to use experimental research to answer research questions. These are apparent in business settings, educational settings, government policy, etc. We’ll just go through a few use cases to help you understand when to use this type of research.
- Conversion rate optimization
For brands that have websites or even brick and mortar stores, there’s no guarantee that the copy, layout, pricing, etc. is ideal. To ascertain the ideal configuration of their website or stores, brands engage in A/B testing.
A/B testing is the process of having a control group and changing an element or group of elements on the website or within the store. A percentage of people interact with the control layout and a percentage interact with the new layout. Whichever one gets more people to perform your desired action is considered the most successful variation.
For example, a brand called Acme Inc. Gets 10,000 visitors to its homepage every month. There’s a CTA on the homepage to sign up for a free consultation. They want to run a research experiment to find the best messaging.
It changes the headline of the page and randomly sends half of its visitors to the page with a new headline and sends half to the original page. At the end of the month, the new version of the page generated 500 free consultations or a 10% conversion rate. The original page generated 350 free consultations or a 7% conversion rate.
- Evaluating new business processes
Instead of rolling out new business processes to an entire organization, which may have a detrimental effect, organizations test it on a batch of their employees first. The majority of the organization – the control group – sticks to the old process. A single department or teams across multiple departments test out the new process. While this is occurring, there is a specific metric that’s being measured whether that’s time to completion, money saved, etc.
At the end of the experiment, the control group is compared to the experimental group. If the changes are significant enough, the new process is rolled out to the entire organization.
- User experience design
Oftentimes, software is designed without initial input from the people that will be using it on a regular basis. This can lead to designs that don’t serve the end-user.
To prevent this, experienced user experience designers create initial mockups, get feedback from their audience, make changes, then do a side-by-side comparison between one version and another.
So, the original version of the design will be shared with a small pool of potential users and the new version will be shared with a small pool of potential users. The version of the design that gets users to perform a specific action more often is considered the better version.
These are a few examples of how experimental research is used in the real world. The thing to remember is that there are multiple groups, a control group, and a variable that can be manipulated at will by the researcher.
Advantages and disadvantages of experimental research
Advantages
- Can use analysis to prove or disprove hypotheses with a high degree of confidence
- It’s not tied to any specific industry so as long as you can design an experimental research study to fit your needs, it can be used.
- Experimental research results can serve as the foundation for more experiments or initiatives in your organization
- Allows you to save a lot of time and money by testing ideas first before implementing them or making decisions
- It’s relatively fast to carry out most types of experimental research studies.
Disadvantages
- It often requires larger pools or participants to get statistically relevant results.
- It becomes exponentially more difficult to test multiple variables against a control. Instead, it’s usually advised to run them as different experiments.
- The researchers will need to know how to perform statistical analysis before they can properly use true experimental research
- If the experiment is set up incorrectly, the results can be skewed in the wrong direction and lead researchers to make the wrong decisions.
Conclusion
Experimental research is an effective tool but it’s just one method in your arsenal. If you want to use it, it’s essential that you create a hypothesis beforehand. If ____________________, then ___________. If this happens then the outcome will be this.
Once the hypothesis is set up, you can then go through the process of creating your experiment and then analyzing the results. From there, you can make informed decisions.
If you have any questions about experimental research, let us know in the comments and don’t forget to share.
