As an initial aside, would you consider putting a spoiler tag in the appropriate titles? I didn't know which, if any, of the two Dalek-posts I could read without being current on my Who.
In the late 1960's, a young Whoopi Goldberg exclamed "Mamma! There's a black lady on television, and she ain't no maid!", referring to Lt. Uhuru on Star Trek. In 1992, Dr. Mae Jemison became the first "black lady" to go into space, also inspired by by Lt. Uhuru. In 1993, she would go on to play transporter officer Lt.jg Palmer in ST:TNG.
As you said, by now, we have the tech demonstrated on Star Trek -- handheld satellite communicators, libraries in our pockets, PADDs/tablets, heck, in some places even a keyboard is "quaint". We have made unbelievable strides in equality. The future, as seen 300 years away in 1967 is almost here, in 2014.
What, then, is there to show for the bright, promising, future? What believable advances can we postulate about in the future? And how will it be relatable to people? What do we do when looking towards a bright, promising, future we see Vinge's Singularity (where technological advance happens so fast its impossible to keep up, even over short timespans)? And by "we", I mostly mean SF writers.
Historians know that the motivations and beliefs of people in 15th century Europe are alien to us. Futurists who know their history have no reason to believe that the motivations and beliefs of people in the 24th (and a half!) century will be any less alien. The amazing progress and change over the last half-century is proof of that.
A lot of the "the future's not awesome" is a way to keep the future relatable: the Singularity didn't happen because some catastrophe reset things so the future is not advanced-beyond-recognition, so we can look at it from now and understand it.
