Show community member ads on Google Maps

Community
1

Sharing my success-story with adapting Google Maps to a category listing adverts from our users. Hope it will inspire others to use maps for their communities. The map looks like this:

image.jpg1890Ă—1055 718 KB

image.jpg1233Ă—1409 714 KB

In our Ukrainian Goat Keeper community, we have a Goat Market category. People put their advertising for free there when then want to sell a goat, a pure-breed goat, a newborn goat kid and so on.

I wanted to make it more visual and let people easily find who sells what close to them. This way they could save on transportation costs, which sometimes may be 50% of the goat price.

I ended up developing a google-map page which reflects the category with advertising listings.

What it does

  • Automatically puts all adverts on the map (uses Google Maps API)
  • Parses images and text out of the first topic post (uses Discourse API)
  • Cleanups text from HTML, removes small images (= smiles and other unrelated)
  • Joins all advertising listings from a single user into a single popup
  • Displays a marker on the map with a counter (up to 9), meaning how many adverts are off
  • Ignores closed and/or archived topics
  • Ignores topics with a very short (or empty) text
  • For every ads, shows a single image only (actually, its thumb), and makes a link saying “N photos”, which will open a gallery viewer
  • Trips the advert text and add the “read more…” link, which opens the topic.

How it works

There is a PHP script that goes through all topics in that category, parses them and builds a list of markers to put on the map, and caches them into database:

  • runs once every 5 minutes to fetch fresh data from Discourse (simply a cron task)
  • uses masterminds/html5 for parsing cooked HTML of topic posts; it removes all formatting and images and then retrieves pure text;
  • the SQL query retrieving information from Discourse is built with Data Explorer in Discourse Admin Panel, and is called via Discourse API
  • uses React to manipulate the map, but it can be achieved same well with pure JS

Disclaimer

We already have a curated database of goat keepers where we store, among other things, their forum username, physical address and latitude/longitude coordinates.

The script matches advertising topic authors to the records in that database by username — that is how we know where to put an advert on the map.

However, this can be easily improved by using the “location” data from user profile in Discourse. If it is filled, Google Maps API can be used to retrieve coordinates for the map.

Possible improvements

  • Add search input; not a trivial task considering a full-screen layout and the need to work well on mobile device screens
  • Parse more images and text from further posts in a topic; it now only parses the first topic; the difficulty here is to filter out non-important advert information and only keep what is to the point
  • Use different markers for sell vs buy adverts; currently not implemented because only up to 5% are “buy” type advertising

Our users fell in love with the maps at first sight! The feedback is extremely positive, so I think the maps will contribute to the community value big time.

Let me know if you like the idea, how would you use it for your community and what other improvements you can think of.

You can hire me to help you with building a similar map page for your community.

21 Likes
2

Here is a simple PHP class I use to call Discourse API:

PHP code 
<?php

namespace Kozomap;

use InvalidArgumentException;
use LogicException;
use RuntimeException;

class ForumApi
{
    public static function call($method, $urlPath, $vars = []) {
        $ch = null;

        try {
            $vars['api_username'] = KOZOVOD_FORUM_API_USERNAME;
            $vars['api_key']      = KOZOVOD_FORUM_API_KEY;

            $url = KOZOVOD_FORUM_URL;
            if (substr($url, -1) !== '/') {
                $url = $url . '/';
            }

            if (substr($urlPath, 0, 1) == '/') {
                $urlPath = substr($urlPath, 1);
            }

            $url = $url . $urlPath;

            if (strpos($url, 'https://') !== false) {
                $port = 443;
            } else {
                $port = 80;
            }

            $curlOpts = [
                CURLOPT_URL => $url,
                CURLOPT_HEADER => 0,
                CURLOPT_RETURNTRANSFER => true,
                CURLOPT_FOLLOWLOCATION => true,
                CURLOPT_PORT => $port,
            ];

            $method = strtoupper($method);
            switch ($method) {

                case 'GET':
                    $url = $url . '?' . http_build_query($vars);
                    break;

                case 'POST':
                    $curlOpts[CURLOPT_POST] = true;
                    $curlOpts[CURLOPT_POSTFIELDS] = $vars;
                    break;

                case 'PUT':
                    $curlOpts[CURLOPT_CUSTOMREQUEST] = 'PUT';
                    $curlOpts[CURLOPT_POSTFIELDS] = $vars;
                    break;

                case 'DELETE':
                    $curlOpts[CURLOPT_CUSTOMREQUEST] = 'DELETE';
                    break;

                default:
                    throw new LogicException('Unknown method to call Forum API.');

            }

            $ch = curl_init();
            if ($ch === false) {
                throw new RuntimeException('Cannot initialize curl_init for Forum API.');
            }

            $res = curl_setopt_array($ch, $curlOpts);
            if ($res === false) {
                throw new RuntimeException('Cannot set curl options for Forum API.');
            }

            $response = curl_exec($ch);
            if ($response === false) {
                $errNo = curl_errno($ch);
                $err = curl_error($ch);
                throw new RuntimeException("Failed calling Forum API, curl_exec returned FALSE. CURL error no. $errNo, message: $err");
            }

            $decoded = json_decode($response, $assoc = true);
            if ($decoded === null) {
                throw new RuntimeException('Cannot json_decode Forum API result.');
            }

            return $decoded;

        } catch (\Exception $e) {

            // Rethrow the exception; might add error logging later
            throw $e;

        } finally {
            if (!empty($ch)) {
                curl_close($ch);
            }
        }
    }

    public static function runDataExplorerQuery($id) {
        if (!ctype_digit(''.$id)) {
            throw new InvalidArgumentException('Data Explorer Query ID is not an integer number.');
        }
        $json = self::call('POST', "admin/plugins/explorer/queries/{$id}/run.json");

        if (!array_key_exists('success', $json)) {
            throw new RuntimeException('Data Explorer Query result does not contain "success" key.');
        }

        if (!$json['success']) {

            if (array_key_exists('errors', $json) && is_array($json['errors'])) {
                $err = $json['errors'][0];
            } else {
                $err = 'No error details provided by Forum API.';
            }

            throw new RuntimeException("Data Explorer Query did not run successfully. $err");
        }

        return $json;
    }
}

For instance, to fetch data returned by query with ID=5 (after you build it in your query explorer), use this:

$data = ForumApi::runDataExplorerQuery(5);

Of course, it will be practical to run data fetching once a few minutes and cache it, and let maps use local (cached) version of the data, otherwise you’ll end up DDoSing your Disccourse instance.

1 Like
3

Brilliant, a practical, clean, well executed concept.
Thank you for sharing.

4

How well would this scale for 1k-2k data points? My concern integrating with Google Maps directly has always been the amount of up-front effort to cull the data down grouped by location, zoom level, and proximity that doesn’t cause too many problems when browsing it.

5

Unfortunately, I never tried that many points.

I’d tackle such a problem in the following way:

  1. Use an existing Google Maps plugin to group markers.
  2. Provide data in a compact normalized mode (e.g. like Data Explorer plugin in Discourse)
  3. Lazy-load data in popups

This way it should not be an issue to handle 1k-2k data points.

6

Gotcha. This is exactly the problem I referenced in the other thread about mapping users. It will fall apart at that scale and there’s not a Discourse plugin that handles it already.

7

It’s not clear, what will fall apart, sorry.
Are you saying Google Maps can’t handle 1-2k markers easily?

8

It’s not about whether or not you can do it at a technical level rather the usability of the data. You really need to do clustering to make it easy to understand: Marker Clustering  |  Maps JavaScript API  |  Google Developers

We’re completely off-topic at this point. The reason I responded here was because I received a message in the other thread about maps of users that pointed me here. My underlying issues are two-fold:

  1. I don’t have a database of these markers. I need to be able to pull them from the user profiles of users in Discourse.
  2. I’m completely unfamiliar with how the back-end Discourse code is implemented. I have neither the time nor the desire to learn it for in order to solve this one problem.

Don’t worry about it. My comments/questions are relatively unrelated to what you’ve implemented. It’s a different problem scenario.

3 Likes
9

I see what you mean.

In your case, I’d build a background task, can be even a simple PHP script, that will fetch users and their IP / Location field, whatever is available. With using existing PHP libraries or API services, should not be an issue to implement.

Then only use your local database with cached user API coordinates to build markers on top of Google Maps. That will be similar to what I did.

For clustering, I think it can easily be enabled by using an existing plugin for Google Maps and not reinventing the wheel.

Hope it helps.
A lot of such things are much easier than they seem, actually. You just start doing it and in 4 hours you get a working prototype, then it gets easier just to tune it up.

3 Likes
10

Was this (project/functionality) improved further? Iinto a plugin perhaps?
Or was dropped?

11

Still same codebase, still works. No changes were introduced to it through last 2-3 years.

2 Likes