Door Hinge pulls rivets out

I just took off my front door (1960 Overlander) and restored the indoor and out door finish. When I put it back I noticed one of the hinges had three very old corroded screws in it. It looked terrible. I decided to pull the screws and take the hinge off and clean it and the aluminum under it . It also needed a new gasket. I riveted the whole thing back on and it looked great. When I put the door back on and closed it it worked great. After I closed the inne rdoor the rivets pulled out of the far three rivet holes. The short pop rivets just did not hold. The three middle rivets that were replaced with new pulled out also. The last three rivets nearest the door are still holding



Under my investigation I realized the hinge is being pulled hard when closing the door with the inner door closed. I have a suspicion that it was not lined up correctly and years ago the previous owner used screws (factory defect possibly). The screws he used were 1/8 in width and very short. I don't know how to solve this problem because I don't want to go with screws and have them be ripped out also. It has a lot of stress on the hinge when closing the door. The other hinge does seem to have that stress. Any suggestion