Converting JSON Data with jq: Powerful Command Line Tool

🎯 What You Will Learn in This Guide

In this guide, you'll learn how to quickly filter, transform, and create new structures of large and complex JSON data using the jq command line tool.
Thanks to this tool, which is indispensable for AI / ML (Artificial Intelligence / Machine Learning) workflows and DevOps processes, you will be able to extract API responses, log files or any information you want from Kubernetes outputs with a single line command.

---

🧠 Technical Summary

Main Technical Topic: Using jq (Command-line JSON Processor)
Solved Problem:
To filter, collect and transform complex data in JSON format quickly and reliably.
Steps Followed:

1. jq setup and testing with identity operator
2. Array iteration (.[]) and data retrieval (.ad)
3. Addition and mapping (map, add)
4. Conditional filtering (select)
5. Combine results into a single JSON structure

---

⚙️ 1. jq Installation

On Ubuntu / Debian Systems

sudo apt install jq

macOS (Homebrew)

brew install jq

Download the binary file from the Windows Official jq page and add it to PATH. To verify the installation:

jq --version

---

🧩 2. Running the First jq Command

Create a test JSON file:

nano denizCanlilari.json

Save the following content:

[
    { "ad": "Sammy", "tur": "köpekbalığı", "istridye": 5 },
    { "ad": "Bubbles", "tur": "orka", "istridye": 3 },
    { "ad": "Splish", "tur": "yunus", "istridye": 2 },
    { "ad": "Splash", "tur": "yunus", "istridye": 2 }
]

Identity Operator (.)

jq '.' denizCanlilari.json

This command shows JSON as pretty print and does not modify the file.

---

🔍 3. Data Extraction and Filtering (.[] and |)

jq '.[]' denizCanlilari.json

Lists each array element separately. To just pull the namespaces:

jq '.[] | .ad' denizCanlilari.json

If you want to remove the quotes:

jq -r '.[] | .ad' denizCanlilari.json

---

➕ 4. Addition and Mapping (map and add)

List all oyster numbers:

jq 'map(.istridye)' denizCanlilari.json

To find the total:

jq 'map(.istridye) | add' denizCanlilari.json

Output: 12

---

⚙️ 5. Conditional Calculation (select)

Let's collect only oysters from creatures of the "dolphin" type:

jq 'map(select(.tur == "yunus").istridye) | add' denizCanlilari.json

Output: 4

---

🧱 6. Converting Data to New JSON Structure

Combine all results into single JSON object:

jq '{
  canlilar: map(.ad),
  toplamIstiridye: map(.istridye) | add,
  yunusIstiridye: map(select(.tur == "yunus").istridye) | add
}' denizCanlilari.json

Output:

{
  "canlilar": ["Sammy", "Bubbles", "Splish", "Splash"],
  "toplamIstiridye": 12,
  "yunusIstiridye": 4
}

---

🚀 Advanced jq Techniques and Usage Areas

High Performance on Large JSON Files

jq --stream 'select(.[0] | length == 2) | .[1]' buyuk_veri.json

Streaming mode processes huge files without loading them into memory.

Kubernetes Integration

kubectl get pods -o json | jq '.items[] | select(.status.phase == "Running") | {ad: .metadata.name, dugum: .spec.nodeName}'

Filtering API Responses

curl -s "https://api.ornek.com/veriler" | jq -c '.[] | {ozellik1: .deger1, hedef: .sonuc}' > egitim_verisi.jsonl

---

💡 Frequently Asked Questions (FAQ)

1. When does jq have an advantage over Python or Node.js?

It provides high speed and low memory usage in small data filtering operations. It is naturally integrated with Shell scripts.

2. What should I do to avoid getting errors in missing fields?

? use operator:

jq '.canlilar[]? | .ozellik?' denizCanlilari.json

3. Is it possible to convert JSON to CSV?

Yes, easily with @csv filter:

jq -r '.[] | [.ad, .tur, .istridye] | @csv' denizCanlilari.json > canlilar.csv

4. Is it possible to do batch processing with jq?

Yes, you can use the -c flag with JSONL files:

cat log_verisi.jsonl | jq -c 'select(.hata == true)'

---

🎯 Result

With this guide, you learned the basic and advanced usage of jq:

Filter, collect and transform JSON data

Increase efficiency in AI/ML and DevOps workflows

High performance with streaming mode on large data sets

💡 You can speed up data processes and automate analysis workflows by trying jq on your GenixNode infrastructure now! ⚙️🚀