SQL Recipes for Socialgist’s BigQuery Datasets: Copy & Paste These Templates

By Aaron Strat on March 5 2024 on

Note: this article was contributed to our site by the Datastreamer team.

In This Blog:

  • Query 1: Monitoring Local Perspectives on China’s Economic Health
    • Sample SQL query and use case analysis
  • Query 2: Chinese Market
    • Sample SQL query and use case analysis
  • What are Analytics Hub datasets?
    • How to import datasets

Sample Query 1: Monitoring Local Perspectives on China’s Economic Health 

SQL Command

Copy & paste the snippet from here: https://thiscodeworks.com/sg-sql-query-1

Query Description

This SQL query searches through Chinese forum posts for mentions of “economy” (in Chinese characters) in content, showing the post’s title, body, publication date, and source URL. 

Advanced Analytics

This sample query is meant as a demo to interact with shared datasets in Analytics Hub. To go beyond keyword filtering you can:

  • Move to Artificial Intelligence (NLP) tools: Filtered datasets can be seamlessly moved into Google Cloud’s NLP tools or Looker Studio for robust visualizations.
  • Apply Human Intelligence: Results can also be manually analyzed by experts or teams to draw nuanced insights and understand context deeply.

Deploy a live pipeline or custom dataset: This public shared dataset has a fraction of total metadata and posts available from Socialgist. You can request a live pipeline or custom dataset for full control over the data (and enriched fields) that you receive. 

A Deeper Look At This Use Case:

Monitoring Local Perspectives on China’s Economic Health 

In the face of global economic fluctuations, a data analytics team sought to understand the ground-level public sentiment regarding the Chinese economy’s health. They delved into forum discussions within Socialgist’s Forums dataset, analyzing posts to build a hypothesis on international market movements informed by authentic opinions.

Methodology:

  • Import Conversational Data: The team was able to easily import Socialgist’s vast repositories of Chinese Forums into BigQuery via a shared Analytics Hub dataset.
  • Data Filtering: SQL queries were structured to sift through millions of posts for keywords related to economic activities, government policies, and public sentiment.
  • Sentiment Analysis: Filtered datasets were exported and fed into NLP tools, where the team categorized posts into positive, neutral, and negative sentiments towards economic conditions.
  • Trend Identification: Temporal analysis was conducted to track sentiment over time, correlating it with major economic announcements or global events.

Application: This real-time sentiment data allowed the team’s client, an international investment firm, to adjust its market strategies more dynamically, mitigating risks and capitalizing on emerging opportunities based on the public’s ground-level economic outlook.

Sample Query 2: Watching the Super Bowl for Consumer Insights

SQL Command

Copy & paste the snippet from here: https://thiscodeworks.com/sg-sql-query-2

Query Description

This SQL query is designed to gather discussions related to the Super Bowl from English forums, to provide a concise and focused dataset that offers a snapshot of public sentiment and discussions surrounding the Super Bowl within the specified timeframe.

Advanced Analytics

This sample query is meant as a demo to interact with shared datasets in Analytics Hub. To go beyond keyword filtering you can:

  • Move to Artificial Intelligence (NLP) tools: Filtered datasets can be seamlessly moved into Google Cloud’s NLP tools or Looker Studio for robust visualizations.
  • Apply Human Intelligence: Results can also be manually analyzed by experts or teams to draw nuanced insights and understand context deeply.

Deploy a live pipeline or custom dataset: This public shared dataset has a fraction of total metadata and posts available from Socialgist. You can request a live pipeline or custom dataset for full control over the data (and enriched fields) that you receive. 

A Deeper Look At This Use Case:

A Closer Examination of the Super Bowl Phenomenon:

The Super Bowl is a cultural phenomenon that captivates millions worldwide. For brands, it represents a unique opportunity to engage with a broad audience through media campaigns. A data team set its sights on uncovering the buzzwords and topics that dominated conversations around this hallmark event. 

Methodology:

  • Import Conversational Data: The first step involved importing Socialgist’s extensive forum datasets into BigQuery, facilitated through a shared Analytics Hub dataset.
  • Data Filtering: Through SQL queries, the team filtered millions of forum posts to isolate conversations related to commercials, halftime performances, and game discussions.
  • Keyword Frequency Counting: The team exported filtered datasets to Looker and Looker Studio to perform keyword frequency counting and highlight the most talked-about topics and themes.
  • Theme Visualization: With dynamic visualizations, the team was able to graphically represent keyword trends and unveiling the narrative threads that captivated the public’s attention.

Application: The insights derived from keyword frequency counting and trend visualization offered the team’s client, a leading advertising agency, a detailed map of public interest and engagement during the Super Bowl. This granular understanding of discussion trends enabled the agency to tailor their future marketing with the topics that resonated with viewers.

What Are Analytics Hub Datasets?

Analytics Hub is a data exchange, run by Google, that enables data assets to be efficiently and securely exchanged across organizations. The primary value is that it allows for a “1-click” import of datasets as a BigQuery table. Data can then be queried, analyzed, or exported in various formats such as JSON. 

Powered by Datastreamer pipelines, Socialgist has released shared datasets on Google’s Analytics Hub, which enables a 1-click import of the world’s conversational data into your BigQuery environment. 

You can also create a direct pipeline from Socialgist to your BigQuery environment (without Analytics Hub) which gives you full ownership and expanded capabilities for managing data flows.

For a guide on how to implement analytics hub datasets, you can view this guide.