Supes Approve $30 Million Contract for Muni Train Control System

3552447195_42aa6b9655.jpg Flickr photo: dylan yee

The Board of Supervisors today approved a five-year, $30 million umbrella purchasing agreement between the San Francisco Municipal Transportation Agency and Thales Transport & Security, Inc. for Advanced Train Control System (ATCS) "improvement services and upgrades." The ATCS system controls the speed, location, and routing of Muni’s light rail vehicles when they’re underground between Embarcadero and West Portal stations. The MTA says the upgrades are necessary to keep the ATCS in working order for the coming decades.

"This umbrella purchasing agreement will help us in our ongoing effort to reinvest in all the infrastructure that keeps our system running," said MTA spokesperson Judson True. "The Advanced Train Control System is one of the most important components of the entire Muni operation and we’re thankful that the Board of Supervisors understood the importance of investing in it to keep it in a good state of repair."

True said Muni hasn’t always made the necessary investments in maintaining the system since it went live in 1998, six years after Muni and the Public Utilities Commission signed off on the original contract in 1992. An ATCS Asset Assessment presented by Thales a year ago also found broad evidence that the system wasn’t being maintained properly by MTA technicians, leading to some components degrading faster than they otherwise would.

Under the "Human Factors" section of the report, for instance, Thales wrote, "(At Wayside) No evidence of Preventative Maintenance being performed. Working knowledge of (ATCS) wayside equipment is incomplete, and with no knowledge of how the (ATCS) equipment operates as a system."

Thales made a series of recommendations, including purchasing a series of equipment upgrades necessary to keep the system in working order and up-to-date, ranging from upgrading ATCS software from OS2 to Windows, to transitioning "to full ATCS and [decommissioning] residual elements of pre-ATCS signal system."

The ATCS system has allowed Muni to greatly increase the number of trains it runs through its tunnels, from fewer than 30 per hour to 60. When the system was originally activated in 1998, however, it brought on a massive meltdown, bringing trains to a crawl through the Market Street tunnel. That installation was performed by Alcaltel Canada, Inc., which Thales acquired in 2007.

Because the ATCS system is proprietary, trade-secret property of Thales (by virtue of owning Alcatel,) the MTA had little choice but to stay with Thales for the maintenance purchasing contract, even if it wanted to enter into a competitive bidding process. The MTA does have some recourse to keep costs down, however, since its board will have oversight over each purchase prior to executing the purchasing agreement, and the agency plans to conduct an independent cost analysis of each purchase order before signing the agreement.

The MTA’s True said the contract is not about maintaining old equipment, but about keeping the system modern for the foreseeable future. "I think that this purchasing agreement allows us to at least move in the direction of always keeping the technology up-to-date. So, it really will be a decision decades down the line of whether to replace the train control system, if there is a new one. The real goal is to keep it in a good state of repair."

Replacing the full system and dumping Thales is not financially realistic in the near term: to do so would cost $100 to $200 million, an MTA report (PDF) to the Board of Supervisors says, and "would require extended periods of early subway shutdowns to provide adequate test windows."

There was little question on the Board of Supervisors that Muni needs to take better care of the ATCS, but it was one of the sobering realities of the public transit business that a 1992 agreement with Alcatel leaves little room for negotiation 17 years later.

  • Jeffrey W. Baker

    The ATCS is a total joke and Alcatel should be sued into oblivion, not rewarded, for their ridiculous implementation. Ever passenger who has to stand on the K as it lurches through the tunnel one meter at a time should get to slap an Alcatel executive with a wet trout. Anyone who has ever stood there while the operator rebooted the system while trying to enter the Duboce portal should receive an all-expenses paid vacation to a city where the streetcars work properly.

  • Peter M

    Will these improvements allow them to FINALLY start loading/unloading multiple trains at once in a station?

  • Something Muni should consider in all future contracting language: Muni must have access to technical data/specifications and must make all technical platforms in open-source formats. This sounds too much like the proprietary Next-Muni data contract. No one is allowed to even scrape the data that is supposed to be publicly available. I know of more than a few programmers that desire to create useful mash-ups with the Next-Muni data and could have provided thousands (perhaps a million) dollars worth of “mined” operational data free of charge. Although contracts with open source clauses may be more expensive in the short run, in the long run it saves Muni money.

  • Alex

    As for ATCS capabilities, go watch sfmunicentral.com, I dare you. I did, and I counted 32 outbound and 34 inbound trains passing through Civic Center from 7am to 8am on October 1st. That’s a far cry from 60. One was not in revenue service, and three others were dependent upon cab signaling*. About half of the trains were only one car, the other were two car. Keep in mind that pre-ATCS MUNI ran two or three car trains through the tunnel all the time.

    * That cab signaling system that Alcatel/Thales wants to get rid of(!) is what lets the trains run fairly fast through the tunnels when the Alcatel crap fails.

    Note that because the MTA has decided to prohibit “double berthing” outbound in West Portal (they still do it *ALL THE TIME ON THE INBOUND S IDE*), my PM commute gets delayed for up to 15 minutes between Forest Hill and West Portal. This doesn’t count the four or five outright ATCS failures (both spontaneous emergency brake applications near Church and Castro stations, and simply shutting down stretches of track) that delayed my commute in September. The Alcatel ATCS never worked properly in the first place, why the hell are we going to give them $36 million (what Alcatel is currently demanding) to fix it?

    Adding insult to injury are the accusations from Thales about how incompetent the MTA staff is. Given how uncooperative Alcatel/Thales was in getting their product installed and training anyone, it’s no wonder their product is decaying so quickly. If you look through Thales’s current marketing propaganda, you’ll see that they claim to have an “open architecture, system modularity, standard interfaces and commercial off-the-shelf data communication components”… but hey, proprietary and open are synonymous, right?

    Of course, if you read the linked PDF you’ll see all sorts of glowing hyperbole attempting to justify the existence of ATCS in the subway. Increased flexibility? They don’t run the S-Castro Shuttle on a regular basis, because it creates problems for the rest of the system. The platform signs still don’t remain in sync with the trains actually going by, and still can’t indicate a proper destination for the S or any short runs. And we still can’t double berth like ya could with the old train control software. ATCS allows the T to enter the subway? Sure, because Da Mayor’s consultants designed that portal around ATCS. Trains entered at Duboce and West Portal for years without ATCS screwing everything up. And the L/M/N run two car (previously three) trains, where the T only runs one car trains.

    Thank you Alcatel/Thales for making my commute so much more obnoxious on a regular basis.

  • Alex

    Ugh. Anti-spam bits ate my post.

    @Seth The data are available, unencrypted from NextBus… albeit somewhat unintentionally. If your friends have good ideas, they should implement them. Worst case, they take the app down. If the Routesy ordeal is any indication, common sense will prevail.

  • NextBus information might not be legally scraped, but you can get the monitor video feed from Central Control (which is the actual ATCS screens used to monitor the subway updated every 5 seconds) if anyone is interested in scaping a really bad video feed based loosely on Pong.

    http://www.sfmunicentral.com/

  • Alex

    @Jamison, a bunch of the NextBus info is available with some effort including vehicle coordinates, speed, heading, arrival predictions, stop predictions, stop coordinates, route paths, run numbers, timestamp of the last gps info, and when LRVs change routes, etc.

    There’s lots of potential there, and quite frankly, it’s about time someone forced the issue… especially as Newsom flaps his gums about open data.

  • Ajay

    @Alex : where is this information? Gaving keeps talking about “open data”, but the data that’s really useful is locked away. How did Routesy get its data anyways? Last I heard in the summer Routesy had been refused the data. Did they strike a deal with Nextcrap ?

  • Alex

    It’s all streamed to the Live Map that’s available on nextbus/nextmuni.com.

  • There’s some background on the routsey saga in the stories linked from here:

    http://sfappeal.com/news/2009/10/hey-you-remember-routesy-right.php

    In summary, the MTA has reasserted their ownership of bus arrival data. I still have no freaking idea where routesy gets at this data, though – I had just assumed they were scraping the nextmuni site.

    FWIW, the MTA is providing a feed to Google, though, and Google’s high-tech “CSV” formats are open:

    http://code.google.com/transit/spec/transit_feed_specification.html

    Moreover, Google gets its info from a zip file which is published via public HTTP access:

    http://maps.google.com/help/maps/transit/partners/participate.html#host

    This is route data, different from the real-time data, of course.

  • Ajay

    @Alex: “It’s all streamed to the Live Map that’s available on nextbus/nextmuni.com.”

    In other words: not accessible to the public. It is public data, and should be accessible to the public. If NextBus has done a deal with MUN for exclusive access to the data, then MUNI should say so. MUNI keeps claiming there is no such deal, and continues to hide the data.

    This smacks of corruption.

  • Alex

    @Ajay Hardly. Don’t confound publicly accessible with intentionally accessible. The data aren’t encrypted. Sure, it’s not intended to be accessed by the public, but the info *is there*. My point is that if you (or anyone else) wants to make an app using this data you should. Push the issue now, while Newsom is crowing about open access.

    I managed to make a small app that pulls the data on a regular basis and converts it to an easier to parse XML format. It’s not rocket science. If you want to do something with the data, do it. Worst case you end up in a Routesy like situation where you’re legally obligated to take the app down*. Best case, Newsom and the MTA are forced to deal with this issue and hammer out more friendly contract terms.

    * Note, of course that this isn’t legal advice and I’m not a lawyer. If you’re concerned about your liability talk to a lawyer, make friends with the EFF, etc.

  • Alex

    Also, I think perhaps a bigger issue is that the MTA doesn’t have any interface for publishing this information. When I’ve pulled data it’s been directly from NextBus. This means that data for pretty much all of their customers (such as AC Transit) is easily accessible. While I agree that government data ought to be open and such, not all of the customers are likely to be as “enlightened” as Newsom/MTA.

  • Ajay

    @Alex : I understand what you’re saying; but then Nextbus can easily thwart your attempts by changing things around. You’re at the mercy of Nextbus. I’d prefer not to build anything that involves reverse-engineering the data, when MUNI claims the data is in the public domain!

    As for having an interface to publish this information: put it in a CSV file somewhere! If they want to get fancy, all they have to do is ask, and I’m sure a dozen volunteers will help them out. This area has the densest population of techies, and MUNI is hamstrung with how to make the data public?

  • Alex

    Ahhh… except, no, it wouldn’t be so easy for NextBus to change out the serialization stuff. Even if it were easy for NextBus to break third party apps, they certainly haven’t tried that yet in the face of Routesy. In fact, if they did try that I suspect there’d be a fair amount of backlash. I don’t know how Routesy is gathering its data, but they are certainly not using any interfaces designed for public consumption.

    As for publishing the data as a CSV… let’s say that there aren’t any problems with using CSV for anything meaningful (there are). So you’ve got a CSV. Then what? The MTA likely has no infrastructure in place to p publish this information in real-time. Yes, there are any number of not so complicated solutions. But that’s not what the MTA bought from NextBus. Compare this to the some of the other data that falls under this openness policy like the GIS data. Not only is it static (thus easier to publish), but the data are presented in the format in which they were created. All told, this is a lot less work than fiddling with the NextBus data. Hell, look at sfmunicentral.com. That was originally setup as a debugging tool, not a publicly visible tool.

    Lots of stuff about the MTA is less than ideal, but why should that stop you from developing an app? Quit the whining, and start a proof of concept app.

  • Ajay

    @Alex : what serialization are you talking about? You’re expecting developers to screenscrape Nextbus’ pages. Screen-scraping is fragile, to say the least.

    And then there’s the problem of hammering Nextbus’ servers. If your application has 1000 users, and you want to update the schedule every 5 seconds, then you’re hitting Nextbus.com 200 times/second. Can they handle that? Will they consider it a DOS? I don’t want to get sued to find out!

    As for the file format: CSV is just a format. Use XML. Use JSON. It doesn’t matter, as long as it’s unambiguous. The data you’re looking for is basically the route and position of each vehicle that’s plying a route; its state (stopped/boarding?); the previous stop; the next stop. It doesn’t have to be “realtime” in the hard sense; as long as it is updated every, say, 5 seconds, a snapshot would be fine.

  • Ajay

    @Alex : you wouldn’t happen to be Alex Orloff of NextBus, would you? Just wondering.

  • Alex

    First off, take a look at the NextBus live map and see how it works. Play around with it a little. I’m not talking about screen scraping at all, I’m talking about using the Java applet to decode the data for you, and (unless you want to write your app in Java or bundle a Java app with your app) act as a proxy.

    Take a look at how the “live map” and other NextBus web apps work. They all update at least once a minute. Even if you were distributing an app that hit the NextBus servers directly, and simply worked via screen scraping, how is this different from what their apps are already doing? Using a proxy is potentially more suspicious, but even then, probably not even a blip on their radar considering how many other customers NextBus has. Routesy hasn’t gotten sued yet, are you planning on something far more popular than Routesy?

    As far as the MTA goes, neither snapshots every five seconds nor real time data are static. What infrastructure does the MTA have to update data regularly like that? My guess is none. That is almost certainly a bigger issue than whether or not they can find someone to simply make the data available.

    And, no, CSV is not quite the same as XML or JSON. It’s not particularly well structured, and does indeed lead to ambiguous situations. It’s a lousy interchange format, full stop.

    And, no, I’m not affiliated with NextBus in any way shape or form.

  • DS

    Here’s an idea (well, several) to totally thwart, confound and pi$$ off the MTA and other agencies who aciently think train arrival data is theirs. Note that this idea will not likely work where cell phone signals cannot get in or out, unless WiFi is exploited.

    Get riders to anonymously or publicly subscribe to a cell tracker service. Subversives types and activists and anyone who intensely feels they have nothing to hide might be willing to sign up. Now, make an app that polls their phones. The subscriber should be able to turn it on and off, but if even 4 or 5 riders on each train are in the program, then some inormation will still be available. Actually, a rider should be able to just opt in and out randomly, and not be required to “subscribe”. This means even international tourists with working phones could join in and influence the system even tho they might be considered “outsiders with no say, no vote…”

    Now, if the transit authorities think they own vehicle arrival information, they will be sorely eye-punched into reality. They cannot own or control cell data of riders. Only in a police emergency or terror incident might the system be disrupted. Any real intent trouble maker is not going to be thwarted by cries of data proprietary status.

    The contributing rider would need to:

    – dial a short code to opt in or out
    – enter the vehicle number (this gives QC so that the devs can block trouble makers who enter ghost of fake car numbers in an effort to undermine the hactivist/public-spiritedness of the program)
    – enter route and direction
    – punch out by walking thru a threshold atop the stairs, or entering a short dial code
    – indicate whether or not returning or waiting or switching trains so as to not confuse the hactivist arrival predictor.

    The devs need to carry beacons or transponders in the blind to QC test the system. Later, the MTA should be FORCED by voters to carry generic, vetted-dev accessible transponders to act as an audit to MTA. (Some of these idease come from of my rage that the 54 and other buses would just “drop out” and then out of rage that my calls for the whereabouts were met by “it’s on the way”; “trust me, it’s 5 minutes away”, and other responses far from realility. So, while the buses are not my point here, the trains are in more managable tracking territory, especially since they cannot just emerge from 280 & Geneva and go into revenue service as if the driver arrived after playing around…)

    I am sick and tired of the system not announcing N-Juda arrivals yet the booth displays will indicate. But, riders waiting to HEAR of the N-Judah have to wait until one is only 4 minutes away, or go back topside to find our or call the booth from the courtesy phone.

    Also, i talked with a platform supervisor/attendant who told me the software cannot handle 3 trains. This is insane, that alcatel and any current owner of the software could hold MTA over a barrel and not provide for lengthening the N-Judah to 3 cars. N-Judah is horribly over crowed. It should be illegal for the trains to be that packed. Other lines (J/L/L/M) seem have an abundance of cars picking up riders. I end up riding to Embarcadero FROM Powell just so that around 6 or 630 i can get a seat because my bag is too heavy to stand 30 minutes to my destination. MTA and SF riders should class-action sue Alcatel/Thales a$$e$ off to upgrade the software to get us to 3 cars and to thin out the standing bodies count and to load and unload multiple joined cars along the entire length of the platform.

    Also, the AC units need to ratchet up during flue seasion. Vents need to evacuate the fouled air from overcrowed trains.

ALSO ON STREETSBLOG

Derailment Shuts Down Muni Metro Service in Twin Peaks Tunnel

|
Riders hustle from an inbound shuttle bus to Castro Station. Photo: Michael Rhodes A train derailment in the Twin Peaks Tunnel disrupted Muni Metro service between Castro Station and West Portal Station today, extending into the evening commute. The middle section of the second car of an outbound L-Taraval train came about a foot off […]

Plans for Muni Cuts Prompt Campos to Call for MTA Audit

|
Supervisor David Campos. Photo: Michael Rhodes The Board of Supervisors doesn’t get to vote on Muni service cuts or worker layoffs, but today Supervisor David Campos exercised one of the options the supervisors do have for influencing Muni by calling for an audit of some of the MTA’s practices. Campos’ call for an audit came […]

Supes Avalos, Wiener Clash on Equitable Spending Strategies for Muni

|
Supervisors John Avalos and Scott Wiener are sparring over how new revenue for transit should be spent to benefit the Muni riders who need it most. With tax measures proposed for the 2014 ballot that could significantly increase transportation funds, Avalos introduced a charter amendment yesterday that would “require the city to prioritize investments to address existing […]