Cities don't stand still. Going back at least to WWII, U.S. cities have been radically altered again and again. Economic restructuring has been part of it, as urban areas have shed manufacturing in favor of the so-called service sector: FIRE (Finance, Insurance, Real Estate) and Tourism (restaurants and hotels plus retail and entertainment). Transportation changes have played a big part too, with the suburbanization of the 1950s-60s fueled (literally) by the interstate highway system and intraurban freeways, and the inexorable expansion of private cars at the expense of public transit. The populations that occupy various neighborhoods in cities, once relatively stable for generations, have moved away, leaving behind spaces whose character has changed with the arrival of new city dwellers, whether from other countries or just elsewhere in the U.S.
It's a long story, and every neighborhood in every city has its own tale to tell. During the past generation a populist opposition to urban gentrification has emerged. It probably starts with the bitter struggles to prevent the 1960s "urban renewal" programs from displacing whole populations (in San Francisco's Fillmore it became known as "negro removal," a precedent well-remembered by those now opposing the Redevelopment Agency in Bayview/Hunter's Point). But during the real estate booms of the 1980s and again during the dotcom boom at the end of the 1990s, right through the historically unprecedented housing bubble that finally popped in 2008, many progressives have worked to confront the forces of gentrification.
Gentrification as a term tends to conflate different "facts on the ground" though. Sometimes it defines a process of social displacement, usually class- or race-based, wherein a poorer population is forced out by rising prices and the steady influx of new residents who can pay those prices. To acolytes of the market, this all seems perfectly reasonable and fair, and the idea that there should be some kind of social restraint on such "efficient" "self-organizing" market mechanisms is anathema. To leftists and housing activists committed to defending the downtrodden and the poor, this system is a thinly disguised process of ethnic cleansing most of the time, and when the outcome isn't blatantly racist, it's still another chapter in a long saga of the rich screwing the rest of us.
Those of us who lean towards this latter way of seeing things are enjoying some schadenfreude today as the Lembi Group, one of San Francisco's most notorious landlords and exploiters of tenants, is sinking under a mountain of debt  it incurred during the frenzied market conditions that only recently subsided.
On the other hand, folks fighting the displacement dynamics of the real estate markets in cities have sometimes fallen into a weird cul-de-sac where they seem to think keeping things seedy and decaying is a good thing, as if that were a way to ensure community stability. I can't cite anyone's public declarations to this effect, but I've certainly heard many friends and comrades tsk-tsking disapprovingly when they see someone painting their building, or putting in sidewalk tables and flowerpots, or any of a number of street-level neighborhood improvements. The sidewalk depaving  and gardens I wrote about in January earned that response from some radicals I know too. I suppose the feeling is that if such improvements begin, it's only a matter of time before the Devil of Displacement rides in behind the ferns and wrought-iron ornamentation.
David Byrne, the New York musician (once of Talking Heads fame), has a new book out called "Bicycle Diaries ," in which he travels to many cities around the world and the U.S., bringing his bike to help preserve his mental (and physical) health while touring. In his "Diaries" he ruminates on many interesting questions of the role of art, history, urban design, and decries the role of freeways in destroying inner cities and waterfronts among other things. But in one curious part he visits Pittsburgh, PA and has this to say about a clear-cut process of gentrification:
"About four years ago when I was here, [a friend] told me how the Heinz family was intent on bringing life (and eventually urban living) back into the downtown of this former industrial giant. Sometimes a rebirth can be started in one neighborhood and then it spreads to the surrounding areas--if they're not cut off or isolated. Artists move into a former factory district and soon cafés and grocery stores follow. A music club opens, a gallery and a bookstore. Developers turn the warehouses into luxury condos and the process begins again, somewhere else."
In other parts of the book Byrne is quite critical of the dynamics of modern capitalism and the results of unfettered market life on the quality of living it leaves in its wake. But here he embraces the building blocks of gentrification, a version of a life-cycle for urban neighborhoods. True enough, this has happened in many places, sometimes aided by the participation of the poorer residents who struggled to clear vacant lots and start community gardens, or who started street festivals when times were harder that have since become charming ethnic attractions.
To my thinking, the problem is not the efforts people make to improve their physical spaces. Of course there's always an issue of "taste," and what is cool and chic for some is cheezy and offensive to others. Some folks might be glad to see an influx of new cafés, while others would prefer the neighborhood remained forgotten (and thereby open to exploration and unmediated interventions) and "unimproved." The real issue is the right of folks who live in an area to make it their home, to have a sense of stability and comfort in their own communities. In the U.S., and in San Francisco, if you don't own property, you have no stability. With the property-trumps-all logic always hovering over city neighborhoods, tenants and the poor are regularly displaced when inflationary dynamics begin, especially if there is no rent control to stabilize their right to remain.
Conflicts arise predictably though, when an incoming population of hipsters, artists, gays, etc., are openly hostile to the population that is being displaced simultaneously to their arrival, often poor, black or latino, and dependent on the underground economy to sustain themselves. The wealthier new arrivals are naturally targeted for harassment and sometimes crime by those who see them as both invaders and insensitive boors with a disproportionate sense of entitlement.
I'm part of the CounterPULSE art/community space  at 9th and Mission in San Francisco, where I curate public Talks  on three Wednesdays a month. When we opened in 2005, the neighborhood had a lot of vacancies in the wake of the post-dotcom bust, and we felt lucky to find a place we could afford and get a long-term lease near to BART and transit. Within a block there are also numerous public agencies serving the transient poor and homeless, and the corner in front of CounterPULSE has a bus stop for both the always exciting 14-Mission, as well as several SamTrans lines, mostly used by working class commuters. By 2007, our place had been tagged countless times and our windows had been thoroughly wrecked by teen vandals with etching acid they used to graffiti the glass. Homeless addicts were often sprawled on our sidewalk, at or near the bus stop, for hours during the day, and our front doors were open toilets during the night. We worried about our vulnerability, especially at first, but over time we realized that there wasn't much actual danger, just the day-to-day reality of living in a central city area with a lot of down-and-out folks living on the streets.
A couple of years ago in September we set up out front to enjoy PARK(ing) Day  and were joined by dozens of neighbors from the buildings on our block. Together we decided to launch a neighborhood association (it has since gone into limbo) called "Planktown Neighbors," a name we chose to signify the fact that we were on a stretch of Mission that had originally been a plank toll road in the early 1850s. In our discussions we struggled to define our goals (to make our streets more beautiful, more comfortable, and safer, esp. vis-a-vis the car-dominated 9th and Mission Streets), but to be as inclusive as we could be. We were in no position to solve the homelessness drama on our two-block stretch of Mission, but we didn't want to be another NIMBY-ish group of small businesses and cultural organizations who called the police to shoo away "undesirable people." So for starters some of the group decided to invest in large planters to help beautify the block, and we all vowed to put more effort into cleaning the sidewalks and getting to know the people in and around our buildings. We all thought it would be smart if we could get enough of a street transformation under way we might be able to get a city grant, and we could pool resources to hire local street people to help maintain our new trees and sidewalk gardens.
This is the dilemma of urban evolution as we live it today. Private property rules this society, and the notion of a public commons, or any sense of a shared public fate, is as weak as it can be. To make gentrification--or even just improvement--something that benefits everyone and not just the lucky few who already have most of the wealth, is the task that we face. How do we ensure that EVERYONE has a decent place to live, enough to eat, and the services they need to cope with the demons they face? How should social stability be valued and preserved AGAINST the rapacious logic of private profit and the market? We haven't asked ourselves these questions much lately, and we'll have to if we want to put an end to the repetitive cycles of displacement, resentment, and racism that plague the normal ebb and flow of human communities in San Francisco and nationally.