# DataPulse MY

> DataPulse MY is an open-source trust layer for Malaysian public data. It publishes
> a machine-readable manifest of 122 official datasets, fresh health metadata, licence
> and attribution for each, and status badges, an RSS feed, and a dashboard so teams
> can assess and reuse official sources from data.gov.my, BNM, DOSM, DOE, KKM, KPDN,
> and MET Malaysia.

DataPulse MY does not replace official sources; it documents what is available,
whether it is fresh, how the schema behaves, and collection quirks. It serves
journalists, civic technologists, data engineers, and AI systems that need stable,
machine-readable dataset-health metadata. This index is forward-compatible agent
infrastructure and does not imply adoption by any LLM provider or improved citations.

## Model Context Protocol (MCP)

DataPulse MY exposes a **read-only MCP server** so AI agents (Claude Desktop, Cursor, Cline, etc.) can query the 122-dataset catalogue natively.

- **MCP endpoint:** https://mcp.data-pulse.my/mcp
- **Transport:** Streamable HTTP (POST-only, single endpoint)
- **Auth:** none required (all 5 tools are read-only)
- **Source:** reads the same `datapulse.json` + `health/latest.json` served from this repository; no separate API

### Tools

| Tool | Use when |
|---|---|
| `search_datasets(query, licence?, source?, limit?)` | Find datasets by topic, agency, or licence |
| `get_dataset(dataset_id)` | Full detail + health for one dataset |
| `find_stale(max_age_hours?)` | Find datasets whose health is not green |
| `get_provenance(dataset_ids[])` | Citation-ready metadata for one or many datasets |
| `find_by_licence(licence)` | Enumerate what's available under a given licence |

### Resources (read-only context)

- `datapulse://index` — lightweight list of all 122 datasets with namespace (read first)
- `datapulse://licences` — licence summary (110 CC BY 4.0 + 12 OGL)
- `datapulse://{dataset_id}` — full manifest entry for one dataset (on-demand)

### Connect from Claude Desktop

```json
{
  "mcpServers": {
    "datapulse-my": {
      "transport": "streamable-http",
      "url": "https://mcp.data-pulse.my/mcp"
    }
  }
}
```

### GTFS-aware discovery

The `transport` namespace contains 16 static schedule ZIPs and 14 realtime
vehicle-position protobuf feeds. Use `search_datasets("ktmb")` or another operator
or place name to discover feeds, then `get_dataset(dataset_id)` for static calendar
freshness and route/stop/trip counts or realtime vehicle counts and feed timestamps.
The `datapulse://index` resource exposes each dataset's namespace for lightweight
transport filtering.

The public endpoint is live at `https://mcp.data-pulse.my/mcp` and serves all 5 read-only tools over the 122-dataset catalogue. Deployment and service details for `datapulse-mcp` are in [`docs/mcp-deploy.md`](https://r3dz4r.github.io/datapulse-my/docs/mcp-deploy.md).

## Manifest

- [datapulse.json](https://r3dz4r.github.io/datapulse-my/datapulse.json): Full manifest — 122 datasets, official URL, licence, attribution, refresh cadence, geographic coverage, namespace, and health-report link. Machine-readable.
- [JSON Schema](https://r3dz4r.github.io/datapulse-my/datapulse.schema.json): Machine-readable schema for the manifest.

## Health

- [Latest health snapshot](https://r3dz4r.github.io/datapulse-my/health/latest.json): Every dataset's current status, HTTP status, Last-Modified metadata, parsed content freshness date, freshness signal source, record counts, and check time. Machine-readable.
- [RSS feed](https://r3dz4r.github.io/datapulse-my/feed.xml): Dataset health changes with status-tagged entries and RFC 822 publication dates.

## Trust posture

DataPulse MY distinguishes between:

- HTTP reachability (URL returns any response)
- Browser-required access (Camofox needed to render the page)
- Schema validity (record count within expected tolerance)
- Freshness (Last-Modified header or parsed content date within declared cadence)
- Unknown freshness (`unknown-freshness`) when HTTP and content shape pass but neither freshness signal is available

A dataset is marked `fresh` only when the applicable direct-access, schema, and
freshness signals pass. See `health/latest.json` `_trust_summary` for the current
distribution and the count of signals that could not be extracted.

## Datasets

- [Malaysian Fuel Prices](https://r3dz4r.github.io/datapulse-my/data/fuelprice.md): Open Government Licence (Malaysia); weekly.
- [ePerolehan Tender Notices (DIIKLANKAN)](https://r3dz4r.github.io/datapulse-my/data/eperolehan-diklankan.md): Open Government Licence (Malaysia); daily.
- [PriceCatcher (Daily Grocery Prices)](https://r3dz4r.github.io/datapulse-my/data/pricecatcher.md): Open Government Licence (Malaysia); monthly.
- [BNM Daily Exchange Rates (0900)](https://r3dz4r.github.io/datapulse-my/data/exchangerates_daily_0900.md): Open Government Licence (Malaysia); daily (weekdays, 0900 MYT).
- [BNM Daily Exchange Rates (1130)](https://r3dz4r.github.io/datapulse-my/data/exchangerates_daily_1130.md): Open Government Licence (Malaysia); daily (weekdays, 1130 MYT).
- [BNM Daily Exchange Rates (1200)](https://r3dz4r.github.io/datapulse-my/data/exchangerates_daily_1200.md): Open Government Licence (Malaysia); daily (weekdays, 1200 MYT).
- [BNM Daily Exchange Rates (1700)](https://r3dz4r.github.io/datapulse-my/data/exchangerates_daily_1700.md): Open Government Licence (Malaysia); daily (weekdays, 1700 MYT).
- [MET Malaysia Weather Forecast](https://r3dz4r.github.io/datapulse-my/data/met_weather.md): Open Government Licence (Malaysia); daily.
- [DOE APIMS Air Quality (Hourly API)](https://r3dz4r.github.io/datapulse-my/data/doe_apims.md): Open Government Licence (Malaysia); hourly.
- [DOE RQIMS River Water Quality (Continuous)](https://r3dz4r.github.io/datapulse-my/data/doe_rqims.md): Open Government Licence (Malaysia); hourly.
- [DOE MQIMS Marine Water Quality (Manual)](https://r3dz4r.github.io/datapulse-my/data/doe_mqims.md): Open Government Licence (Malaysia); monthly.
- [KKM iDengue Weekly Dengue Cases](https://r3dz4r.github.io/datapulse-my/data/kkm_idengue.md): Open Government Licence (Malaysia); daily.
- [OpenDOSM Crime by District & Type (Annual)](https://r3dz4r.github.io/datapulse-my/data/dosm_crime_district.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Monthly CPI by State & Division](https://r3dz4r.github.io/datapulse-my/data/dosm_cpi_state.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Annual Real GDP by State & Sector](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_state_real_supply.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Quarterly Real GDP](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_qtr_real.md): Creative Commons Attribution 4.0; quarterly.
- [OpenDOSM Annual Real GDP by Supply Sector](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_annual_real_supply.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Monthly Trade Headline](https://r3dz4r.github.io/datapulse-my/data/dosm_trade_headline.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Monthly CPI Inflation by Division](https://r3dz4r.github.io/datapulse-my/data/dosm_cpi_inflation.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Monthly Trade by End Use (BEC)](https://r3dz4r.github.io/datapulse-my/data/dosm_trade_enduse_bec.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Quarterly Labour Force Statistics](https://r3dz4r.github.io/datapulse-my/data/dosm_lfs_qtr.md): Creative Commons Attribution 4.0; quarterly.
- [OpenDOSM Quarterly Labour Force Statistics by State](https://r3dz4r.github.io/datapulse-my/data/dosm_lfs_qtr_state.md): Creative Commons Attribution 4.0; quarterly.
- [OpenDOSM Annual Employment by Sector and Sex](https://r3dz4r.github.io/datapulse-my/data/dosm_employment_sector.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Population by State](https://r3dz4r.github.io/datapulse-my/data/dosm_population_state.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Nominal GDP by Supply Sector](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_annual_nominal_supply.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Quarterly Nominal GDP](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_qtr_nominal.md): Creative Commons Attribution 4.0; quarterly.
- [OpenDOSM Quarterly Real GDP (Seasonally Adjusted)](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_qtr_real_sa.md): Creative Commons Attribution 4.0; quarterly.
- [OpenDOSM Annual Nominal GDP and GNI](https://r3dz4r.github.io/datapulse-my/data/dosm_gdp_gni_annual_nominal.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Monthly Core CPI Inflation by Division](https://r3dz4r.github.io/datapulse-my/data/dosm_cpi_core_inflation.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Monthly CPI Inflation by State and Division](https://r3dz4r.github.io/datapulse-my/data/dosm_cpi_state_inflation.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Monthly Producer Price Index](https://r3dz4r.github.io/datapulse-my/data/dosm_ppi.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Annual Labour Force Statistics](https://r3dz4r.github.io/datapulse-my/data/dosm_lfs_year.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Monthly Labour Force Statistics](https://r3dz4r.github.io/datapulse-my/data/dosm_lfs_month.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM Monthly Trade by SITC Section](https://r3dz4r.github.io/datapulse-my/data/dosm_trade_sitc_1d.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM IPI for Export-Oriented Divisions](https://r3dz4r.github.io/datapulse-my/data/dosm_ipi_export.md): Creative Commons Attribution 4.0; monthly.
- [OpenDOSM IPI for Domestic-Oriented Divisions](https://r3dz4r.github.io/datapulse-my/data/dosm_ipi_domestic.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Monthly Interest Rates](https://r3dz4r.github.io/datapulse-my/data/dgm_interest_rates.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Quarterly Federal Government Revenue](https://r3dz4r.github.io/datapulse-my/data/dgm_federal_finance_qtr_revenue.md): Creative Commons Attribution 4.0; quarterly.
- [data.gov.my Quarterly Federal Operating Expenditure](https://r3dz4r.github.io/datapulse-my/data/dgm_federal_finance_qtr_oe.md): Creative Commons Attribution 4.0; quarterly.
- [data.gov.my State Government Expenditure](https://r3dz4r.github.io/datapulse-my/data/dgm_state_finance_expenditure.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Monthly Money Aggregates](https://r3dz4r.github.io/datapulse-my/data/dgm_money_aggregates.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Monthly Currency in Circulation](https://r3dz4r.github.io/datapulse-my/data/dgm_currency_in_circulation.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Monthly Payment Systems](https://r3dz4r.github.io/datapulse-my/data/dgm_payments_systems.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Monthly Payment Instruments](https://r3dz4r.github.io/datapulse-my/data/dgm_payments_instruments.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Monthly Payment Channels](https://r3dz4r.github.io/datapulse-my/data/dgm_payments_channels.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Annual Interest Rates](https://r3dz4r.github.io/datapulse-my/data/dgm_interest_rates_annual.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Annual EPF Dividend Rates](https://r3dz4r.github.io/datapulse-my/data/dgm_epf_dividend.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Monthly Vehicle Registrations by Type and Fuel](https://r3dz4r.github.io/datapulse-my/data/dgm_vehicle_registrations_type_fuel.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Daily FPX Transactions](https://r3dz4r.github.io/datapulse-my/data/dgm_payments_transactions_fpx.md): Creative Commons Attribution 4.0; daily.
- [OpenDOSM Annual Births by State](https://r3dz4r.github.io/datapulse-my/data/dosm_birth_state.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Deaths by State](https://r3dz4r.github.io/datapulse-my/data/dosm_death_state.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Maternal Deaths by State](https://r3dz4r.github.io/datapulse-my/data/dosm_death_maternal_state.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Marriages by State and Sex](https://r3dz4r.github.io/datapulse-my/data/dosm_marriages_state.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Hospital Beds by State and Hospital Type](https://r3dz4r.github.io/datapulse-my/data/dgm_hospital_beds.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Healthcare Staff by State and Staff Type](https://r3dz4r.github.io/datapulse-my/data/dgm_healthcare_staff.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Daily Blood Donations by Blood Group and State](https://r3dz4r.github.io/datapulse-my/data/dgm_blood_donations_state.md): Creative Commons Attribution 4.0; daily.
- [data.gov.my Infant Immunisation Coverage](https://r3dz4r.github.io/datapulse-my/data/dgm_infant_immunisation.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Sexually Transmitted Diseases by State](https://r3dz4r.github.io/datapulse-my/data/dgm_std_state.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Daily PeKaB40 Health Screenings by State](https://r3dz4r.github.io/datapulse-my/data/dgm_pekab40_screenings_state.md): Creative Commons Attribution 4.0; daily.
- [data.gov.my Malaysian National Health Accounts Expenditure](https://r3dz4r.github.io/datapulse-my/data/dgm_mnha.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Monthly Electricity Consumption](https://r3dz4r.github.io/datapulse-my/data/dgm_electricity_consumption.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Monthly Electricity Supply](https://r3dz4r.github.io/datapulse-my/data/dgm_electricity_supply.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Water Consumption by State and Sector](https://r3dz4r.github.io/datapulse-my/data/dgm_water_consumption.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Water Production by State](https://r3dz4r.github.io/datapulse-my/data/dgm_water_production.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Access to Treated Water by State and Strata](https://r3dz4r.github.io/datapulse-my/data/dgm_water_access.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Daily Public Transport Ridership](https://r3dz4r.github.io/datapulse-my/data/dgm_ridership_headline.md): Creative Commons Attribution 4.0; daily.
- [data.gov.my Monthly KTMB Ridership](https://r3dz4r.github.io/datapulse-my/data/dgm_ktmb_ridership_monthly.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Cellular Subscribers by Plan Type](https://r3dz4r.github.io/datapulse-my/data/dgm_cellular_subscribers.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Prisoners by State and Sex](https://r3dz4r.github.io/datapulse-my/data/dgm_prisoners_state.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Drug Addicts by State and Age Group](https://r3dz4r.github.io/datapulse-my/data/dgm_drug_addicts_age.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Female Representation in Local Authorities](https://r3dz4r.github.io/datapulse-my/data/dgm_local_authority_sex.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Female Representation in Parliament](https://r3dz4r.github.io/datapulse-my/data/dgm_parliament_sex.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Monthly Marine Fish Landings by State](https://r3dz4r.github.io/datapulse-my/data/dgm_fish_landings.md): Creative Commons Attribution 4.0; monthly.
- [data.gov.my Crop Area and Production by State](https://r3dz4r.github.io/datapulse-my/data/dgm_crops_state.md): Creative Commons Attribution 4.0; annual.
- [data.gov.my Public Education Institutions by District](https://r3dz4r.github.io/datapulse-my/data/dgm_schools_district.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Household Income, Malaysia](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_income.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Household Income by State](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_income_state.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Household Income by District](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_income_district.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Poverty, Malaysia](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_poverty.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Poverty by State](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_poverty_state.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Poverty by District](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_poverty_district.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Income Inequality, Malaysia](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_inequality.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Income Inequality by State](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_inequality_state.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Income Inequality by District](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_inequality_district.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Household Expenditure by DUN](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_expenditure_dun.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Household Expenditure by Parliamentary Constituency](https://r3dz4r.github.io/datapulse-my/data/dosm_hh_expenditure_parlimen.md): Creative Commons Attribution 4.0; biennial to triennial (survey years).
- [OpenDOSM Annual Population, Malaysia](https://r3dz4r.github.io/datapulse-my/data/dosm_population_malaysia.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Population by Parliamentary Constituency](https://r3dz4r.github.io/datapulse-my/data/dosm_population_parlimen.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Deaths by District and Sex](https://r3dz4r.github.io/datapulse-my/data/dosm_death_district_sex.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Marriages by State, Age, and Sex](https://r3dz4r.github.io/datapulse-my/data/dosm_marriages_state_age.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Fertility](https://r3dz4r.github.io/datapulse-my/data/dosm_fertility.md): Creative Commons Attribution 4.0; annual.
- [OpenDOSM Annual Maternal Deaths, Malaysia](https://r3dz4r.github.io/datapulse-my/data/dosm_death_maternal.md): Creative Commons Attribution 4.0; annual.

## Optional

- [Live dashboard](https://r3dz4r.github.io/datapulse-my/): Human-readable dataset status cards with embedded health and manifest data.
- [GitHub repository](https://github.com/r3dz4r/datapulse-my): Source, samples, contribution guide, and project documentation.
- [Status badges](https://r3dz4r.github.io/datapulse-my/badges/): Per-dataset SVG health badges.