It seems your ex in-laws took out their anger on you by showing they don't care about their grandchildren. I'm sure they must feel so proud of themselves. However your ex should have been the one to ask them, not you. Does he care about your children?