Sweetened Condensed Milk Question

For the impending holiday, I am preparing some pumpkin pies. When cooking pumpkin pies, I unintentionally purchased FAT-FREE sweetened condensed milk.

Will this have a big impact on the finished product? I am searching for enjoyment, not for healthy options. Should I return to the store and purchase the standard sweetened condensed milk?

I truly enjoy making rich, creamy fudge by melting chocolate chips with butter and sweetened condensed milk. The process is incredibly satisfying, and the result is a decadent treat that melts in your mouth with every bite.