I used to work for a federal agency that dealt with this stuff and went to a lot of meetings around the OMB race/ethnic directives.

Personally, given what you say, I would check yes for you and your children.  However, it's all a personal choice.  Some people who are multi-racial or -ethnic check multiple boxes, some check just one ... it's your decision.

According to Wikipedia:

The U.S. Office of Management and Budget currently defines "Hispanic or Latino" as "a person of Mexican, Puerto Rican, Cuban, South or Central American, or other Spanish culture or origin, regardless of race".

I would say you fit that definition. It doesn't have to be 100%.