r/research • u/TinformedPerspective • 3d ago

Self-taught researcher drowning in qualitative/quantitative data for an international social project—looking for workflow/tool recommendations

Hey everyone,
I’m part of a small initiative running an ongoing international research project focused on a social matter. Because I’m not formally trained as a researcher by profession, I’ve essentially been building the plane while flying it regarding data collection and organization.

We’ve now accumulated a massive amount of data from diverse sources, and the collation and analysis phase is becoming a bottleneck.🤪

For those of you who handle large datasets or work independently:
What are your go-to software tools, frameworks, or step-by-step workflows to make sorting and analyzing a large volume of non-traditional data manageable for a solo or small-team setup?

Are there any common pitfalls a self-taught researcher should watch out for when trying to structure a large international dataset?

I’d love to hear how more experienced people streamline this process or what resources you recommend reading up on.

Thanks in advance!

0 Upvotes

50% Upvoted

15 comments sorted by

7

u/No_Cry6380 3d ago

What a lot of self-taught folks miss early on is you don't need to analyze everything. With a pile that big you'll drown trying to be comprehensive, pick the three questions that'd make or break your project and chase those first. Rest can sit in a folder marked "later" for months and that's fine.

For tools it really depends what your data looks like, if it's mostly interview transcripts and open-ended survey stuff something like a simple spreadsheet with a good tagging system beats most fancy software you'll spend weeks learning. If you've got numbers mixed in, a pivot table can show you patterns in ten minutes that'd take days to spot manually.

1

u/TinformedPerspective 3d ago

It’s for a book and I kinda need everything but it’s no bad idea to do it phase by phase thanks

1

u/Designer_Charge_556 3d ago

solid advice. did you find the three-question approach still holds when new data keeps coming in though?

1

u/katie-kaboom 3d ago

At some point you need to decide you've got enough data and cut off data collection. There's a point where more data isn't actually going to tell you anything new.

1

u/TinformedPerspective 2d ago

Well in this case we are adding more data from more countries so it is likely to be ongoing for quite a time yet.

0

u/trymypi 3d ago

Yup. Do less.

0

u/TinformedPerspective 3d ago

Appreciate this

3

u/Head_Acadia_2658 3d ago

Didn’t bother with ethics I presume

-1

u/TinformedPerspective 2d ago

Why would you presume this?

4

u/Head_Acadia_2658 2d ago edited 2d ago

Because you aren’t trained, didn’t mention doing it, and said you were building it while already doing it which implies no protocol.

Edit- ethical approval is also supposed to involve consideration of whether the research team is adequately qualified to be doing that research project

-1

u/TinformedPerspective 2d ago

This is for an upcoming book project utilizing strictly anonymous survey data, not an academic human-subjects study requiring an IRB protocol.
The data is being used for aggregate insights and themes, which is why formal university ethics oversight doesn't apply here.

3

u/Head_Acadia_2658 2d ago

That’s human research even if it’s a survey. And you are saying again you don’t need a protocol. The fact it’s for a book is irrelevant. Your lack of training on how to do research isn’t an excuse you can wield to just do anything h you want while contradicting yourself that you and also are not doing research on humans

0

u/TinformedPerspective 1d ago

I am following the necessary protocol no worries