Genç yaşlarında, [[Afşar]] komutanı [[Nadir Şah Afşar]]'ın yanında, ordu hizmetine girmiştir. Nadir Şah'ın yanındayken, devletle ilgili incelikleri öğrenmeye başlamıştır. [[Muğan Kurultaı]]'nda Nadir Şah'ın aleyhine oy verince, Nadir Şah tarafından [[Horasan]]'a sürgün edilmiştir.<ref name=":0">[[Qarabağnamələr|Qarabağnamə]], [[Mirzə Adıgözəl bəy]], 1950, s. 54</ref> Penah Ali, buna rağmen kendisine bağlı kişilerle geri dönmüş ve bir süre Karabağ Dağlarında, bir süre ise [[Şeki]]'de gizlenmiştir.
The capital of the khanate was moved three times to strengthen the Panah Ali's power in the Karabakh. The Bayat fortress, built in 1748, was the khan first residence. "In a short period of time, external walls were constructed, ditches were dug out, and the bazaar, the bath and the mosque were build." Craftsmen from surrounding areas were re-settled into the castle. "Many of residents of the area and even residents, especially craftsmen, of the [[Tabriz]] district and [[Ardabil]], moved into the Bayat castle with their families, having heard about Panah Ali Khan's success, governance and mercy."<ref>Mirza Jamal Javanshir (1847), History of Karabakh, Baku, 1959, p. 68</ref>
Strengthening of Panah Ali khan's power faced resistance from other khans (e.g. Khan of [[Ganca]], Khan of [[Shaki Khanate|Shaki]]) and from [[meliks]] of [[Nagorno-Karabakh]]. The struggle between the Karabakh khan and [[Haji Chalabi Khan]] of Shaki, one of the most powerful feudal rulers of the [[South Caucasus]], started in 1748. Haji Chalabi Khan wishing Panah Ali khan's power not increased further, allied with the Khan of [[Shirvan Khanate|Shirvan]] and surrounded the castle of Bayat. The allies for the whole month unsuccessfully tried to capture the capital of the Karabakh khanate. The Shaki and Shirvan khans withdrew, incurring huge casualties and failing to accomplish the mission. Haji Chelebi Khan was forced to admit: "Until now Panah Khan was raw silver that was not minted. We came, minted it, and returned." <ref name="adigozel"/> Same quote from another Karabakh historian of 19th century, Mirza Yusif, reads: "Until now Panah Khan was merely gold, we came and minted a coin from that gold."<ref>Mirza Yusuf, Tarihi-Safi, 1856</ref>