Harry was born in Boston's Back Bay June 4, 1898, the son of one of the richest banking families in an area full of rich banking families. Tycoon J.P. Morgan was his "Uncle Jack", and it was taken for granted that Harry would follow in the footsteps of his successful ancestors and spend his days enlarging the Crosby millions. Unfortunately, the first World War intervened, and Harry returned a changed man. He fell in love with Polly Peabody, a wealthy woman in her own right; unfortunately, she was already married. Her divorce, and subsequent trans-continental affair and remarriage to Harry in 1922 scandalized Boston society, but the worst was yet to come. Crosby re-named Polly Caresse, because he thought that Polly was too prim and puritanical a name for his new bride. She was to keep that name until her death in 1970.

Harry was not only a gambler, a womanizer, a drinker and smoker-he was also an incurable spendthrift and he spent a good part of his adult life making lists of ways to improve himself morally. None worked, however; later in life he seemed to resign himself to his dissolute lifestyle. He once sent a telegram to Boston: PLEASE SELL 10,000 WORTH OF STOCK. WE HAVE DECIDED TO LEAD A MAD AND EXTRAVAGANT LIFE. He would organize carriage races in the Paris streets and throw beer-filled balloons from high windows at passers-by.

Harry was obsessed with the sun-his poetry and diaries abound with references to it-quatrains, hymns, sonnets, all dedicated to the solar disk. To him it was a symbol of perfection, freedom, heat, enthusiasm, and destruction. He seems to have focussed more and more on the latter as time went on, but throughout his writing are references to dissolution and suicide. He seems to have made suicide pacts with nearly every woman he was involved with. He was to keep his word. On December 10, 1929, he was found in bed with a neat little .25 caliber hole in his right temple next to a young woman with a matching hole in her left. Harry was still clutching the pistol in one hand, the girl in the other. Even the case-hardened detective noted, "what a beautiful man".