My advice would be that there's merit in starting off your career in a large, serious, engineering-oriented company - not a startup or SME, but it doesn't have to be FAANG or similar either. For me it was Siemens, a traditional engineering company. It wasn't glamorous but it was sensible and positive. These places have graduate programmes and actively develop their new junior staff in a structured way. They also have serious processes and methodologies that serve you extremely well in industry, even if in some contexts later in your career you choose to ignore them.
This also looks good - from an employer perspective, it shows that you have been accepted into and prospered somewhere reputable. You come out of formal learning as an inexperienced blank canvas and it's very likely that you will have learnt how to do things well at a big corporation, whereas someone who has only ever worked at a startup may not adapt well to more risk-averse engineering or corporate processes. It's possible that all their experience has been writing software in a very loose, cavalier fashion.
But, once you've established this, you're not bound to stay forever. As you identified, there's a fear that you must remain in that big name world to stay employable, but it's a myth. You can learn a lot from a startup culture or an SME, provided it's not your only influence. And there is an inverse - you can get institutionalised by staying in one place or one type of environment. I left Siemens and worked for an SME no one has ever heard of, and it's there that amongst other things I was able to blend development work with customer facing consultancy in a way that would have been very difficult in a larger company with more mature, inflexible role definitions. It taught me a lot, years into my career. And now without any difficulty getting hired out of there, I'm back at a household name corporation where I apply the different things I got from my different experiences.
Your outlook is extremely smart, insightful and self-aware. Treat your career with respect that the future version of you will be grateful for, and keep your options open, but do what makes you happy and fulfilled.