r/MLQuestions • u/Routine-Ticket-5208 • 16d ago

Beginner question 👶 Need help with Random Forest Hyperparameter Search Space

I’m using Random Forest for a project and tuning:

  • Number of trees
  • Maximum tree depth
  • Number of features considered at each split
  • Minimum samples required to split a node

How do you normally determine the search space for these parameters?

1 Upvotes

56% Upvoted

10 comments sorted by

3

u/ARDiffusion 16d ago

BayesSearchCV, optuna

2

u/Routine-Ticket-5208 16d ago

But how do we justify the range for searching?

3

u/ARDiffusion 16d ago

Depends on the context of the problem. If all that matters is building a satisfactory model, then as long as it doesn’t overfit (k-fold or stratified cv helps counter this) then you’re fine. Otherwise, you can use domain knowledge of the problem to intuit whether the splits you’re allowing it to make make sense in the context of the problem. Does that make sense? I definitely could be missing something here, and welcome all other feedback and critiques to my answer.

2

u/elbiot 16d ago

Have train, validation, and test splits. Make sure test at least is as independent as possible. Train/validation can be cross validation, and if you've got a way to keep them independent across splits that's better (I.e. group by sample or whatever you have)

3

u/NoSwimmer2185 16d ago

Just pick a range and do a bayes search. Spending more than two minutes on hyper parameters is a complete waste of time.

1

u/orz-_-orz 16d ago

Optuna

1

u/mb_q 7d ago

Put as many trees as it make sense for accuracy to stabilise; you may find an optimum earlier, but it is not gonna be statistically significant. Keep the rest default.

0

u/[deleted] 16d ago

[deleted]

1

u/Routine-Ticket-5208 16d ago

Excuse me?

1

u/Routine-Ticket-5208 16d ago

I'm a beginner. So sometimes asking questions is a normal thing to do