Mechanism of fracture in rubber materials for high pressure hydrogen vessels is highly affected by the hydrogen dissolved in rubber during hydrogen exposure. One of the origins of this effect is thought to be the change of higher order structures of the rubber materials by high pressure hydrogen dissolved in these materials. In this work, we focused the dispersion states of zinc stearates in the rubber materials and studied the higher order structure change of the six kinds of rubber materials, EPDM and NBR with and without the silica and carbon fillers, before and after high pressure hydrogen exposure by using FT-IR spectroscopy. We gave the evidence of the higher order structure changes after high pressure hydrogen exposure and these changes depend on the chemical structures of the rubber materials.