Out of the box LLMs are trash, but you can use their APIs to great efficacy on large amounts of data to find needles in haystacks. Complex needles, not just strings of text like any sophisticated search.

You’re right that anything it finds needs to be gone over by a human, but it HUGELY reduces problem sizes when used intelligently.