I also just finished 'unkindness of ghosts'and agree with all you say! It's so good!

My Vorkosigan opinions are unpopular i guess but i really strongly don't advise skipping 'Warrior's Apprentice'. I think it's such an important baseline for who Miles is and how he fits in to that world that you're missing a lot of context for him going forward. (Also I just love it madly for all that parts of it are maybe silly or clumsy -- I also have to confess i haven't read 'Brothers' or 'Mirror Dance,' I went straight from TWA + Mountains of Mourning to Memory and I keep rereading TWA instead of filling in the gaps. But I truly think the three act structure is great and truly memorable and what LMB says about it in her forward to the collected edition really stuck with me