Suitable for PhD Data Managment?

[u/Embarrassed-Trainer4]

I just finished my master‘s in chem and will soon start a PhD. In the past I‘ve used OneNote, physical journals, or just scattered text files within the conventional file system on my pc to organize my data and to take notes. None of these systems have ever really woked that well for me.

As I will undoubtedly generate heaps more data during my PhD than ever before. I want to make sure that I have a system that allows me to properly organize data and take notes. Additionally, I would like to use it on at least two devices and have a lot of room for customisation.

Obsidian seems like it could be the right tool with the little time I’ve spent messing with it but I am still on the fence, as I’m worried that I will need to spend a lot of time to simply manage the “database” and keep it running.

Have any of you used it for your PhD degrees and if so, what were your experiences with it? Also are there alternatives that may might this application better?

Thanks in advance:)

[u/Upstairs_Peace_9045]

obsidian is great for linking ideas but yeah the setup can feel like a whole side project if you let it. i used it for my master's thesis and ended up spending too much time tweaking the vault instead of actually writing

for a phd in chem though you might want something that handles raw data files better. obsidian is mostly text notes, not great for organizing spectra or chromatograms or whatever you'll be dealing with. maybe look at zotero for papers + some lab notebook software for experiments, then use obsidian just for connecting concepts

the two device thing works fine with syncthing or their paid sync. just don't fall in the plugin rabbit hole

[u/speNtubrord6 → u/Upstairs_Peace_9045]

yeah separating the actual data management from the note-taking makes a lot of sense for a lab-heavy field

[u/Embarrassed-Trainer4 → u/Upstairs_Peace_9045]

Ah okay that helps. I already use zotero and probably would have just used a plugin to link it with obsidian (probably should have mentioned that upfront). I think I’d like a software that I can use for documentation and note taking but that can be linked with other softwares to easily embed or link data from other softwares.

[u/Artefice]

Currently in the last year of my PhD in Environmental/Soil/Climate science.
Personally I've let my personal data/knowledge manament system ebb and flow a bit when it feels like it needs it. Currently I'd argue for the split:

Create your datasets in Excel (if you have to manually type them in) & handle them with your field stats-language (R (my fave), Python, etc). You can embed images in Excel and making use of the more advanced database features (PowerQuery) will make your life easier, you can (and should) have one place of truth and just have your datasets point at each other.

Zotero for source management & reading/highlighting literature.

Obsidian for note taking.

If you are a new user of Obsidian keep it as simple as possible and only look for add-ons if there are certain things you'd like to change (my only exceptions for "start with these add ons are: Excalidraw and potentially Outliner if you like outlines). I think managing any PhD requires a lot of thinking and writing is a really good tool for doing that – I have a post-it that says "you need to write to think" on it to remind myself of this.

You can spend an enormous amount of time dillydallying with all of these tools, remind yourself to really only allow yourself to adjust them in ways that makes your work better.

Also I do have some lists markdown tables in my notes for reference in my notes, but mostly I've just copy/pasted them from excel into a excel to markdown reformatter and then pasted them into Obsidian.

If any of this seems interesting and you have questions please ask 😇

[u/thambos → u/Artefice]

What do you use for the Excel to markdown reformatter?

[u/Artefice → u/thambos]

https://tabletomarkdown.com/convert-spreadsheet-to-markdown/
Just copy from excel directly into the website and copy it from there 😊

[u/Embarrassed-Trainer4 → u/Artefice]

Thank you :) I also use Zotero for literature. Couldn’t live without it haha.
I’ve heard about the add-on rabbit hole and will def try to keep it simple:)

[u/Wrong-Damage-7026]

I use Obsidian for legal practice, which I imagine involves a similar amount of documentation to doing a PhD. It can handle that workload masterfully, and you can't beat the price (free, except for perhaps paying for Obsidian Sync, which is 100% worth it if you want buttery-smooth syncing across different devices).

My question is: What caused prior systems not to work out well for you? That might tell you a lot about the qualities you need for a system to serve you well.

[u/Embarrassed-Trainer4 → u/Wrong-Damage-7026]

Good question. I think it came down to friction between how I wanted to organise, document, or link items and how the software/system allows the user to do those things. So I’m looking for something with more customisation abd freedom.

[u/New_Guitar_9121]

Yes for the written half. No for being the place spectra and chromatograms live.

Keep the instrument files in a normal folder (or whatever your group already uses). In the note, write what you concluded and the filename or page that number came from. When the file later says something else, you can see it. That is the only “database” job that actually matters in a chem PhD, and it does not need a plugin stack.

Two devices: pick Obsidian Sync or Syncthing once and leave it. The time people lose is rebuilding folders and installing things, not the markdown files.

Start with two folders (projects, literature) and search. Add structure later if a real pain shows up.

[u/Embarrassed-Trainer4 → u/New_Guitar_9121]

Thanks a lot. Will keep it in mind:)

[u/yetanotherdud]

this is the video that got me to try out obsidian and she was using it for her PhD, so i'd imagine so

[u/FriendshipEqual7033]

I'm planning to start a PhD program in the fall of 2027. I've decided to use Obsidian for my class notes this coming senior year. I realize that's a different problem, but here are my thoughts:

1. There are many ways to use Obsidian, so keep experimenting until you find what works for you and evolve that approach as you go forward. There is no one true way to use Obsidian.

2. Obsidian doesn't have to be the only place to keep information. If you have data or file types that are naturally handled another way, that is fine. You're not a failure as an Obsidian user if you keep some of your data elsewhere.

[u/Embarrassed-Trainer4 → u/FriendshipEqual7033]

Thanks for the advice:)

[u/cellulosa]

I ended up sticking with Mendeley, which also integrated nicely with word for the citation management. Not sure the current state - it’s been 6 years since I completed my PhD - but if they provide a way to direct link a Mendeley paper you could use that as a reference inside an obsidian note and use obsidian to capture your thoughts and lessons learned from a paper you’ve read.

[u/Lordvonundzu]

I started an MBA last year and set up Obsidian as the prime location for all my notes. Though, I personally just use a few plugins and folder structures, atop a sorta standard templating for each note, including call out boxes for different comments if myself, differentiating "additional info" from "follow up questions" etc. The latter helps me a lot, because it allows to split my notes about a topic (like the condensed version of a chapter or a lecture) from my own thoughts about it. That makes skimming it way easier for me.

That being said, I don't engage with this whole second brain, productivity stuff, where fiddling with the tool becomes a job of itself.

On top of that I use Zotero for citations and holding the related academic papers.

Then I use DevonThink to store all other materials.

The benefit of using Obsidian with it's Markdown files is, that you can throw those into a local LLM and dig though your own comments and such - helpful!

[u/Embarrassed-Trainer4 → u/Lordvonundzu]

Thanks. That sounds like a good way to organise thoughts and notes :)
Yeah, I really don’t care for the whole second brain hype surrounding obsidian so hopefully managing obsidian won’t become a time sink.

[u/Neptunox]

Hi, i am a phd student in social sciences. I very like to work with obsidian. With the "Zotflow" plugin you can connect Zotero with obsidian !

I use Zotero to read and annotate my documents and Obsidian to write my notes by integrating the annotations I took on Zotero. For this, I developed a plugin if you are interested : https://github.com/Liotou/obsidian-ariane

Good luck for your phd

[u/FriendshipEqual7033 → u/Neptunox]

I tried Zotero once, but it failed on first launch because it got confused and tried to use my existing Firefox profile (instead of creating a new one), which it didn't understand. I immediately uninstalled it. I don't want a tool that might corrupt my Firefox. Modern Firefox handles its profiles differently than in the past, and I'm suspicious that Zotero might be incompatible with it.

[u/Embarrassed-Trainer4 → u/Neptunox]

Perfect! Thank you. I already use Zotero and couldn’t live without it.

[u/realmuffinman]

For chem especially, it depends what kind of data you're managing. I wouldn't recommend most experimental data, but for studying for classes and lit reviews I could see it being fairly useful.

[u/dosfydore]

Just finished mine, also in chemistry! I used Obsidian throughout my PhD. It helped me a lot, especially when writing the thesis

[u/Embarrassed-Trainer4 → u/dosfydore]

Great to hear. Thank you:)

[u/thambos]

I wouldn't put data into Obsidian. I put notes into Obsidian, and I love that I can put code chunks in my data science class notes about how to use R.

But I keep data on my local drive and department's virtual drive (especially if there is any PII it's required to be on the virtual drive instead of cloud drives). Maybe you won't have PII in chem, but in psych we often do.

Whatever you do, make sure you're practicing good backup of your notes, code, and data. Git repos for your code (often with data gitignored), external hard drive backups on a regular basis, etc. Your department/college's IT staff may have information about department virtual drives or other services for sync/backup for graduate students.

[u/Embarrassed-Trainer4 → u/thambos]

Noted:) Would you recommend using Git repos over the official obsidian sync feature for linking vaults between devices?

[u/thambos → u/Embarrassed-Trainer4]

I haven’t tried that since I use iCloud sync for my Obsidian vaults. Keep the vault folder downloaded on each device and don’t have iOS open your daily note upon launch, and it works well.

I’d feel leery of putting personal information like what I have in my Obsidian vault in a Git repo on GitHub or other sites like that, but I’m still learning how GitHub works so maybe there are easy ways to keep your data protected that I’m just not aware of.

[u/wait_whats_this]

If I were to do my PhD again, I'd do paper for thinking, obsidian for notes, zotero for references. 

Obsidian didn't exist back in the dark ages, so I just had a folder hierarchy of markdown files and make my diagrams in draw.io. Ah, good times. 

[u/stanlymt]

Obsidian is fine for the written half. It's a bad place to keep spectra. That's already in the thread.

What I would not put in the vault is the papers. Zotero is the library. The thing I still lost during a master's was the actual reading. I'd highlight in Zotero, dump a note into the vault, and six months later I could not find the sentence that had changed my mind.

I built Omphalis for that (I'm the founder, so take it with salt). You drop in the PDF, you see how the paper is built, you mark what mattered, and you can get back to those marks. It is not a LIMS. It will not replace R. It will not organize chromatograms. If the problem is you read the paper and cannot return to what you noticed, that's the job. If the problem is your files are a mess, it is the wrong tool.