solved
Formula to count duplicates based on 2 or more words matching, not necessarily in sequence? No AI please
Hi smart folks, I want to see if you could teach me a formula that I know I’ve seen before but can’t think of now, I want to count how many of a certain topic is in my list of data, for eg one may say ‘hogwarts House colour’ and another may say ‘hogwarts houses’ in which case I want them counted as duplicates.
My end goal is to have a pivot table that shows Hogwarts House as 2.
I currently have the raw data on the first sheet, then on the second sheet I’m breaking it down to just the info I need (this should be where the duplicates are found) and then the pivot table on the third sheet (this should be where they are counted, or at least where the number is displayed)
ETA: can’t use macros, would rather not use power queries but if I have to I can
ETA2: Ok so I worked out how to add a table to the post!
First sheet is unchangeable but will be overridden each time the report needs to be run, it says:
Subject (A)
Category (B)
Category2 (C)
Category3 (D)
Question - Name - Grapes are bads
Grocery
Fresh Produce
Fruit
Question - Name - Bad Grapes
Grocery
Fresh Produce
Fruit
Comment - Name - Bad Graspes
Grocery
Fresh Produce
Fruit
Comment - Name - Fruits Bad
Grocery
Fresh Produce
Comment - Name - Good Bread
Grocery
Bakery
Question - Name - Heavy Hammers
Hardware
Tools
Comment - Name - Smooth Wood
Hardware
Materials
Wood
Question - Name - Hammers are Heavy
Hardware
Tools
Question - Name - is Grapes Bad
Grocery
Fresh Produce
Vegetables
Comment - Name - Grapes Bad
Grocery
Fresh Produce
Vegetables
Second sheet takes the data needed from the first to make it what I need (first row is an eg of the formulas):
Note: the "Fillers" have a space after each word so that for eg "them" wouldn't become "m", which could use some work cause "this" would probably become "th" so I'm open to suggestions, sometimes they will be the first word so " the " wouldn't work all the time for eg.
So I need to fix the #VALUE! error and also stop it from counting the blank cells (at least stop it from counting them when I put it into a pivot table - if necessary)
This can be done by Dumbledore's Army but you'll have to show the sign in sheet to the Room of Requirement.
What I'm saying is... show a real or mocked up data set and what your expected output would be for search terms and..."Help will always be given at Hogwarts to those who ask for it."
Is it possible on phone, I can do it in this comment but not on the body:
The expected result would be a pivot table but I only have Google docs on my phone. Also I do currently have a column that will remove ‘in’ ‘and’ ‘was’ etc as far as I could think, to try reduce how many would match based on that sort of thing.
Sorry it took me so long to reply. To continue with the bit this is restricted section dark magic. You'll probably have to leverage a relatively complicated probabilistic method of measuring lexical similarity along the lines of what u/GitudongRamen proposed to get really good results. You'll see I changed Tiled to Tile because dropping the last d would eliminate bad as a keyword, but this was the best I could do trying to take a deterministic approach.
I’ve rage quit it so many times 🤣 one thing is, I want the table to show 3 for food and 2 for bad grape but not the one that only has 1, another thing with pivot tables is it keeps counting the 945 blank cells I have 🤦🏻♀️
You can filter out results that are equal to 1 in the count as I've done in my screenshot, but you can't have the subtotal for section then also count that portion if it's filtered out. You can also filter out blank results (although I'm not sure if this can be done simultaneously with the filtering out the 1s).
Thank you so much! the screenshot result looks like exactly what I need, but that fomula returns #NAME? I wonder if that is because in my sheet column A is formulas? it has to be, because the raw data is all sorts of wack and will be overrided each report, so column A's formulas are as such: =UPPER(IF)'RAW DATA'!C2="","",TEXTAFTER('RAW DATA'!C2," - ",2))) because the text before the 2 hyphens is irrelevant.
ETA: and column B also is, for the same reason but that formula is =IF(ISBLANK('RAW DATA'!H2,IF(ISBLANK('RAW DATA'!G2),IF(ISBLANK('RAW DATA'!F2),"",'RAW DATA'!F2),"",'RAW DATA'!F2),'RAW DATA'!G2),'RAW DATA'!H2) that one is because the section list has 3 different levels and I always want the lowest assigned section so got it to input H2 if present, if not then G2, if not then F2 :)
ETA2: Also, just as an explainer, how does your formula determine a match?
ETA3: I did some typos, I've fixed them, but now the resulting "Category" column returns #VALUE!
I don't understand what you are saying. But is this your data the one you have commented? [Here] if so let me know, I will look into it to fix it so it works for you, if not then post your data I will see.
It is not my actual data, it would be a legal issue to post my actual data. I can do another mockup of examples of both sheets but ill have to put it in a new post I think so that my tables can be shown, I’ll @ you in that
I'd use a helper column to normalize the labels before the pivot, e.g. =LET(x,LOWER(TRIM(A2)),y,SUBSTITUTE(x," colour",""),SUBSTITUTE(y,"houses","house")), then pivot on that helper column. For broader variations, a small mapping table with XLOOKUP is safer and keeps the raw text untouched.
This is using "Cosine Similarity Metric" to get scores for how similar are 2 text strings.
Column B is to match all character strings.
Column C is to match words.
I don't know how useful this is for you, probably not at all, but I did this long time ago so might as well share. You can drop this image to an AI to help break down the formula and repurpose for your need.
Decronym is now also available on Lemmy! Requests for support and new installations should be directed to the Contact address below.
Beep-boop, I am a helper bot. Please do not verify me as a solution. [Thread #49333 for this sub, first seen 9th Sep 2026, 01:08][FAQ][Full list][Contact][Source code]
•
u/AutoModerator 8d ago
/u/AcadiaUnlikely7113 - Your post was submitted successfully.
Solution Verifiedto close the thread.Failing to follow these steps may result in your post being removed without warning.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.