Guest list troubles

So my FH and I are trying to plan our Manila wedding and we have had a hard time trying to figure out what to do about how many people we would like to have at our wedding. We orginally wanted to have a smaller wedding with 75 guests but I don't think it's possible.

I have a HUGE family with lots of aunts, uncles and cousins and just about all of the cousins have kids.

My FH's family is a difficult situation. His real father is not around, his first step dad pretty much raised him but is more involved with his new wife and kids but is still close with those family members, his second and current step dad has a huge family who he is close with also.

With family it doesn't leave much room for close friends. So, do you think we have one of those invite all or nothing situations so there's no conflict with people being offended by not getting an event invite?