Hi Arindam,
The original ESP32 Feather features an onboard charging IC for the battery so you can seamlessly switch power sources, you can read more here: Power Management | Adafruit HUZZAH32 - ESP32 Feather | Adafruit Learning System
To measure the battery voltage you’d have to create a voltage divider to step down the batteries voltage sweep to something the ESP’s ADC can handle @Liam120347 has a good guide on setting a similar circuit up here: 12 day project: ESP32 - LED controller - #6 by Liam120347
I’d go with one of the LiPo’s Core stocks and a Featherwing protoboard (so you can solder up the voltage divider).
Liam.