← Blog archive

Open Data · May 15, 2016

Liberating Data from NYC Property Tax Bills

Property Tax constitutes the single biggest source of revenue for the New York City Government, accounting for approximately $22.5 Billion dollars in the Modified FY2016 Budget. That's a lot of money, and I've always been curious about how properties are assessed, how exemptions and abatements are applied, and what the overall property tax contribution is of a neighborhood, a block, or any other chunk of the city.

As it turns out, everything about property taxes is public record, but there isn't really any quality bulk data around about it in NYC. Some aspects of property taxes show their face in PLUTO, including assessed value and abatement, but I learned a few years ago that that's only part of the story, and you can't infer from PLUTO how much a property actually pays in taxes. For that information, you have to look at the bills, which are also public record.

The NYC Department of Finance has an online tax bill lookup tool, where you can search for any property by BBL (Borough, Block, and Lot Numbers) or Address. Every statement and bill is available, going back for several years.

Let's take a look at the bill for 1 Centre Street, a building owned by the city government. The bill for June 2015 is available as a PDF from this URL: http://nycprop.nyc.gov/nycproperty/StatementSearch?bbl=1001210001&stmtDate=20150605&stmtType=SOA

If you scroll down to the 2nd page, you'll see a nicely formatted section called Annual Property Tax Detail, and here is where you'll find all of the goodies, including "the bottom line", or how much that property owes the city for a given tax cycle.

Cursor_and_Jul2015

You can see from the bill that if this building were not city-owned, its owner would have to fork over $7.4 Million a year.  You can also see that this figure is based on the Billable Assessed Value of $69.8M times the tax rate of 10.6840% for commercial properties.  The math is clear, but the source of the Billable Assessed value and the Estimated Market Value are not... but that's another blog post. (It's a dissertation, really... Here's a great article from City&State that talks about several administrations' failed attempts to reform the property tax system, and how two comparable properties can pay very different taxes based on how fast their neighborhoods appreciated in value. The most interesting tidbit is that renters are shouldering more property tax burden than homeowners, because of the way apartment buildings are taxed.)

Here's another bill that shows exemptions AND abatements together, saving this owner over $5,000 due to School Tax Relief (STAR) exemption and the Co-op/Condo Abatement:Cursor_and_Jul2015 2

Back in 2013, I put together a little project to extract these nuggets of data, as I wanted to make a simple web map that would allow you to explore property taxes.  To accomplish this, we wrote a series of scripts to download thousands of tax bills and scrape the text out of them. The text was then parsed, resulting in machine-readable data for each property we had a bill for. We ended up pulling out most of the data for Manhattan (condos excluded), and the rudimentary map is still up today at nyctaxmap.herokuapp.com (this hasn't been updated since, and won't be)  It was always my intention to expand it to the rest of the city, and to share the bulk data, but like so many civic hacking projects, it never happened.

Fullscreen_5_12_16__22_13

Last year, NYC Civic Hacker John Krauss located another gold mine of data in the tax bills, specifically the number of rent stabilized units in an apartment building.  Activists and renter's rights groups had been in search of authoritative data on rent stabilized units, and John figured out that he could get high quality data at the building level by scraping every bill for apartment buildings with over 6 units.  He found that surcharges for rent stabilization revealed a massive concentration of stabilized apartment loss in certain neighborhoods.  Read his blog post about the rent stabilization data here, and check out the raw tax bills and data at taxbills.nyc. There's also a great article at ProPublica about the rent stabilization failure-to-report-but-still-getting-the-tax-break situation.

Ben Wellington (iQuantNY) has also done an analysis of the rent stabilization data that was liberated from the tax bills, showing the net change over time among other things. Read it here.

We teamed up to expand taxbills.nyc to include every BBL in the city, condominiums included, with a goal of assembling high-quality granular data including every exemption and abatement in the city. We used MapPLUTO 15v1 as our master list of BBLs, and the DOF assessment rolls to get an exhaustive list of condo BBLs and to join the condo units to their physical tax lot.

Here's the Data

The data exist in two tables, one with a row for each BBL in the city, containing estimated market value, tax rate, total tax due, etc, and the other containing a row for each individual exemption and abatement found in the bills.  There is a one-to-many relationship between these tables, as many exemptions and abatements can be associated with a single BBL.

http://taxbills.nyc/tax_bills_june15_bbls.csv - 1.1 Million Rows

Columns:
address - street address, collected from the bill
bbl - 10-digit Borough, Block, and Lot ID
ownername - owner name, pulled from the bill
condo - 'lot' for condo lot BBLs, 'unit' for condo unit BBLs, empty for all others
condonumber - corresponding condominium number for condo lot or unit
taxclass - the tax class of the property
taxrate - the tax rate associated with the tax class
emv - estimated market value
bav - billable assessed value
tbea - tax before exemptions and abatements (bav x taxrate)
tba - tax before abatements
propertytax - total tax due (the bottom line)

june15exab.csv - 750K rows

**Columns:
**bbl - 10-digit Borough, Block, & Lot ID that this exemption or abatement is associated with
type - 'exemption' or 'abatement'
detail - text of the exemption or abatement pulled from the bill
amount - dollar amount of the exemption or abatement

These two tables were derived from the very large rawdata.csv on taxbills.nyc.  You can see the exact SQL queries and some other aspects of our methodology on the notes page.

A Few Good Queries

According to the data, the total tax before exemptions and abatements is $34.5 Billion.  This is, in theory, the amount the city could collect if there were no exemptions and abatements. I'd love to see what this number would be if real market values were taken into account citywide.

The total tax due (the sum of all of the bottom lines in the bills) is $21.6 Billion, meaning the total amount of tax exemptions and abatements is $12.9 Billion.   Of course, many of the exemptions are simply due to the land being owned by a government agency (does it make sense for a city government to pay itself tax? Should cities charge property tax to federal government-owned land in their jurisdictions?)

Let's take a high-level look at the exemptions and abatements.

There are 147 unique exemption types. There are 42 exemption types with 10 or fewer occurrences. There are 7 with only 1 occurrence: indust waste facility, prof. maj lea sports, nys med care fac fin, patrol salvage, solar/wind energy, state retire system, ltd profit hsng co:l (Someone map these please! Ok, I'll do it myself, give me a few days)

You can see the full list of all 147 exemptions along with a sum ($) and count column in the table below.  I'll keep my opinions to myself for now and just show you the data.

detailsumcount
park$-1,235,613,8254242
dept. of education$-962,818,6261276
port auth-air termin$-913,424,67339
fed govt land bldgs$-566,423,433121
icip$-512,009,1716427
house of worship$-476,243,4085829
nyc housing auth$-427,602,3661121
hospital$-423,899,214478
421a (20 yr not cap$-334,361,000291
dept real est (dcas)$-326,204,3002317
college-university$-306,038,884298
421a (15 yr not cap)$-279,189,00733571
nyc industrial dev$-254,687,155392
421a (25 yr not cap$-252,078,32719785
battery park auth$-238,561,0433076
new mult dwellings - 421a$-225,805,25915956
nys urban dev (esdc)$-223,169,276265
dormitory authority$-196,179,285156
state lands bldgs$-188,206,157702
health & hospitals c$-187,499,52553
dept small bus svcs$-185,986,785189
j-51 alteration$-176,986,09222283
edc - pilot$-154,825,08762
port auth-world trad$-144,282,8064
bd higher ed (cuny)$-144,110,833121
420c housing$-137,270,9981487
religious-school$-136,316,447625
basic star - school tax relief$-131,481,459321063
charitable$-122,001,6961369
senior citizens homeowners‰Ûª exemption$-118,970,13444436
school-elem,hs,acad$-97,272,864278
mta-nyc transit$-82,620,953237
dept pub work (dep)$-76,593,333856
post office$-76,529,620123
roosevelt island$-70,090,358588
cemetery (private)$-69,699,532191
student dormitory$-62,707,580209
enhanced star - school tax relief$-62,542,22582913
dept of correction$-60,485,4507
hospital staff hsg$-60,265,761403
faculty student hsg$-54,279,562348
dept of sanitation$-53,215,234133
public museum$-50,161,23525
house pres devl adm$-49,520,0111516
spec init pgm (sip)$-49,093,441774
public library$-46,087,925191
dept of traffic$-43,088,377520
nyc educ const fund$-41,645,78326
421a (15 yr cap)$-40,592,3453546
prof. maj lea sports$-40,234,0901
nfp-contemplated use$-37,451,904298
fed hospital$-37,039,5665
police dept$-36,494,205198
dept of soc svc (hra$-36,451,518110
foreign mission$-36,108,239185
united nations$-35,967,69210
nursing home$-35,626,82276
mental-moral improve$-35,484,703289
resid-conv. low manh$-29,054,0912555
museum$-28,863,04059
div of alt mgmt prog$-28,557,5981003
cooper union$-28,356,6034
chartble philan$-27,435,310221
opera house$-26,254,2814
school construc auth$-26,065,44845
state aided pub hsg$-25,844,06425
nyc hous auth-duplic$-25,801,28799
perf arts bldg$-25,471,47111
armed forces$-24,736,2095
health center$-24,220,909117
state hospital$-23,834,60931
urban dev. act proj.$-23,677,6389156
urban renewal$-21,201,190120
dept water resources$-19,665,961191
port auth-bus facilt$-19,318,7325
charitable housing$-19,306,166356
parsonage$-19,213,7981059
fire dept$-18,638,363258
benevolent$-18,069,075144
port auth-indust dev$-16,820,00850
state public works$-16,785,619117
veteran exemption$-15,836,96132132
st assisted priv hsg$-15,079,76618
trust for cult rsrce$-14,637,483250
ida - pilot$-13,208,44448
foreign consulate$-13,102,555102
academy of music$-12,826,2119
police fire$-12,256,97312
un development corp$-12,161,4746
religious-dormitory$-11,549,26652
mta - bridge&tunnel$-11,522,68454
disabled homeowner$-11,221,1034866
mta-lirr/metro north$-11,137,72666
state retire system$-9,863,6491
dept of health$-9,684,50742
mitchell-lama (srt)$-9,301,68181
military$-8,269,9037
hsg dev fund co (srt$-7,919,96530
dwelling from fed$-7,253,37825
salvation army$-6,095,29141
veteran portion fully exempt$-5,557,90010708
veteran portion partially exempt$-5,470,23510708
foreign embassies$-5,459,56649
port auth-bridge/tun$-4,552,95739
non-profit med dent$-3,849,5133
dept of highways$-3,013,48637
421a (10 yr cap)$-2,972,734302
foreign staff hsing$-2,767,13485
literary$-2,648,0139
armory$-2,520,9416
fraternal organiz$-2,236,2425
redevelopment (srt)$-2,156,07416
nys hous fin agency$-2,005,203194
hip center$-1,898,83010
nyc econ dev corp$-1,688,0078
library$-1,589,16313
historical$-1,463,65121
421b (1-2 family)$-1,276,1051384
religious missions$-1,085,09129
a.l.,vfw,cwv,jwv,etc$-1,058,59165
inst of arts sci$-895,2395
theatrical corp$-839,4329
public playground$-612,7554
scientific$-539,9969
borough president$-474,9739
ltd profit hsng co:l$-423,8941
clergy$-382,0941331
cemetery (federal$-265,3042
enf/law/child/animal$-259,4963
bible$-245,9762
vol. fire co.$-228,91613
memorial assn$-211,5296
amtrak (fed subsi rr$-151,2173
ny state power auth$-137,6423
limited div (srt)$-97,8603
supvd. sportsmanship$-67,2382
patrol salvage$-60,9981
nys med care fac fin$-45,9531
ser disabled veteran$-24,3514
crime victims$-21,3816
solar/wind energy$-12,3791
indust waste facil$-10,1501
lighthouse$-9,6463
environ prot ex$-8,4412
port auth - invalid$-9103
fallout shelter$-1532
veteran exemption total$126,567,56210708

Something wonky is going on with that last line "veterans exemption total".  If anyone figures it out, please let me know.

Now the abatements, of which there are 9 distinct types:

detailsumcount
co-op/condo abatement$-419,578,05141111
j51 abatement$-85,370,16333884
residential conversion abatement$-39,932,0142564
industrial & commercial abate prog$-28,433,537429
lease abatement$-17,322,212218
brooklyn bridge park pilot$-4,896,130600
solar elec generating system abate$-4,643,4802179
mitchell lama star benefit$-4,550,181135
icip abatement$-1,180,58984

By joining the data with MapPLUTO in CartoDB, we can visualize the tax footprint of the entire city. (Once joined, you can also use the rich lot-level data in PLUTO to analyze property taxes based on zoning, land use, etc)  I have colored lots with zero tax due as pink, but bear in mind that it's often not so simple and things like Payments in Lieu of Taxes (PILOTs) are not reflected in the tax bills. Condo units are outlined in yellow.

So there you have it... we turned 1.1 million pdfs into a high quality open dataset on NYC property taxes, including all exemptions and abatements.  Data scientists everywhere, go forth and crunch the numbers, and please share your findings on twitter with me (@chris_whong), John Krauss (@recessionporn) and Ben Wellington (@iquantny).  We can't wait to see what you come up with!

This post was migrated from my old WordPress blog. Embedded maps and links to third-party sites may no longer work.

Comments (archived, closed)

  1. ENOTTY ·

    Great work! Very curious about the technical details and difficulties of parsing the PDFs. I know from personal experience that isn't always a trivial task, so and i get discouraged when I see datasets enclosed in thousands or more of PDFs.

    Were you just lucky with these PDFs, or did you have to do some special tricks to parse them?

    1. Chris ·

      Yes, we were lucky in a way:

      The first step is extracting the raw text from the PDFs, which is only possible for certain types of PDFs. The linux command line tool pdftotext does this, and preserves the layout. Next, because every bill has a well-formatted area that shows the calculations and the final amount due, a custom script must be written to do pattern matching and isolate the individual fiehttp://blog.chriswhong.com/wp-admin/edit-comments.php#comments-formlds (pattern matching using regular expressions).

  2. nb8 ·

    Bravo! The city needs more brilliant work like this

  3. Anna Montoya ·

    We'd love to host this dataset on Kaggle for public analysis. You can share the code you've run on it alongside the data so folks can build on your work and discuss ideas. If you're interested, let me know! anna@kaggle.com

    1. Chris ·
  4. Leo ·

    Hi Chris,
    This dataset is immensely useful. I know you've posted the code, but have you considered updating the tax bill CSVs on this page for those of us without python capacity?

  5. Ralph E Yozzo ·

    Does anyone notice this Roosevelt Island exemption? Why does Roosevelt Island not seem to pay any property tax?

  6. Ralph E Yozzo ·

    Also see https://tony.brooklyncoop.org/exemptions for a map of all the exemptions in New York City. Thank you John Krauss for the data.

  7. PS Dhillon ·

    Hey Chris, how have you joined the data with MapPLUTO, I was trying to use open source for getting the same geographical map, but I am kind of struck as BBL columns of both datasets are different or am I missing something?
    Please assist

  8. Lynn Ellsworth ·

    Link to your map is now dead. Can you fix it? Love to see the map....

  9. Eric ·

    Hi there,
    Where did you get the data for the School Tax Relief (STAR) exemption? Was that scraping or did you find a dataset? thanks

  10. Meifeng ·

    Hi Chris,

    I am very interested in looking at this amazing data, but I seem to have problems downloading them. Could you assist on this? Thanks!