After Muhammad & Allah took the momentous step of permitting in Islam sex slavery (Quran 70:29-30 & elsewhere), powerful Muslim leaders across the centuries have kept concubines & established harems.

We read in history that both Caliphs (successors to Muhammad, political & religious leaders of the worldwide Muslim community) and (especially) sultans had multiple sex slaves. As an example, the fourth Caliph Ali (Muhammad's son-in-law), so beloved by Shia Muslims, had (at least) two sex slaves, Humia and Umm Shuaib.
A most helpful book, Why concubines, sex slavery, and harems flourished under the Caliphs and Moguls: The Story of rented wombs, helps us gain better insight on how sexual slavery was conducted, and is quoted below;

1. Just who were the sex slaves?   "A Sultan or Caliph could not take a Turk or an Arab Muslim woman as a concubine, a Muslim concubine was not allowed, this is why they imported slave girls from the Balkans and Eunuchs from Ethiopia." (emphasis added). We learn that being a sex slave was a fate consigned to non-Muslim females.
   
2. Was there any advantage to having a concubine instead of a wife?  "[The Caliphs & Sultans] wanted rented wombs, not familial dynasties where a wife’s family would be a threat, they wanted to be known as the Caliphs and Sultans without any reference from the maternal side, it was a perverse living with reference to paternity only, and you would hardly find any maternal cousins within the Ottoman hierarchy." (emphasis added).
3. How did women end up as sex slaves to the powerful Muslims? "The only women who ended up in the Harem were kidnapped or sold into the slave market just as cattle were....Young girls of ravishing beauty were sent to the court of the Sultan as gifts from leaders of other countries. Others were bought from slave markets, abducted or sold by their families even poorer." (emphasis added).
4. Was life as a sex slave pleasant or difficult? "The harem life was a life in a snake pit and not easy, everyone had an interest in their survival and according to her capacity wove conspiracies to topple others. It was an imprisoned life, a segregated complex of buildings populated mostly by captured foreign slaves."

Some Debate Questions!
So, Muslims refused to enslave Muslim females as concubines, enslaving only non-Muslims in this matter, but how would the Muslims feel if another religion enslaved only Muslims as sex slaves?
Muhammad accepted two females, Mariya and Sirin, as gifts from an Egyptian ruler, and thus we have read (above) without being surprised that Sultans would later do the same thing.  But is accepting another adult human as a "gift" moral or immoral?
Does keeping sex slaves eventually have a perverse effect on the minds of the male Muslim slave-owners?
Finally, can we be surprised that the Islamic State is acquiring female sex slaves in our own century?