dougu1985 2013-03-22 05:03
浏览 65
已采纳

生成人口热图:Mapreduce?

I have a MySQL table containing

  • (100 million) Lat/Lng coordinates of locations in America
  • Number of people living within a square mile radius of that location

Question: After generating and overlaying the heatmap on a Google maps or Openstreetmaps, the number of people living with a square mile radius has to be determined at any point on the map wherever the mouse cursor is positioned at. (Simple averaging using the neighboring data points can be used)

How do you generate such a heatmap? Is it recommended to use Mapreduce?

enter image description here

Initial Thoughts

Heatmap has to be pre-rendered serverside

Downloading all the necessary points onto the browser then generating the heatmap clientside can be a problem: Large number of coordinates have to be retrieved from the database (heavy database load) AND transferred to the browser (large dataset), furthermore the browser have to process the large number of points to generate the heatmap. This will be far too slow, so I suppose we have to pre-render the heatmap serverside and retrieve the heatmap tiles to overlap on the map.

Better Alternative: Process serverside, render clientside

Instead of fully rendering the heatmap server side and serving the image tiles, we can simplify the data by clustering points that are close together into a single point and a weight/bias, then send these smaller dataset of simplified point data (via JSON) to the browser for clientside rendering of the heatmap (using heatmapjs). Sending lat/lng points instead of image tiles will make the application/website more responsive.

This will also allow us to read the heatmap intensity values directly from Javscript and implement the hover popup box (see image above) in Javascript/jQuery. Not sure how to do this if we instead had sent the heatmap tiles to the browser.

Map/reduce?

We probably need to split up the job (processing 100 million data points) into smaller chunks and generate the heatmap across several nodes. This will be done once a month. Having several nodes generate the heatmap makes me think of mapreduce and hadoop, although I have not used them before.

Existing solutions

gheat generates the heatmap on-demand, so it will be too slow for our purpose. However we still need a tile server for the heatmap tiles that we pre-render, maybe we can use an OSM tile server.

  • 写回答

1条回答 默认 最新

      报告相同问题?

      相关推荐 更多相似问题

      悬赏问题

      • ¥15 protel99.SE提示一下弹窗
      • ¥15 银河麒麟v10 执行.run失败如何解决
      • ¥15 如何用Python打开LA文件
      • ¥15 用mysql做一个高校工资管理系统
      • ¥21 Multisim设计用三端集成稳压器设计稳压电路
      • ¥15 tp6,在iframe里的网址通过获取url参数失败
      • ¥15 打开opendaylight查看拓扑,为什么直接就显示了两个Switch,在mininet配置了一个topo,这两个Switch还在
      • ¥15 无法远程连接pgsql
      • ¥15 建立DLNM模型时显示基础基矩阵不一致
      • ¥15 贝叶斯重复测量方差分析