# feat:完成数据采集

.gitignore

@@ -0,0 +1,40 @@
+target/
+!.mvn/wrapper/maven-wrapper.jar
+!**/src/main/**/target/
+!**/src/test/**/target/
+.kotlin
+
+### IntelliJ IDEA ###
+.idea/**
+.idea/modules.xml
+.idea/jarRepositories.xml
+.idea/compiler.xml
+.idea/libraries/
+*.iws
+*.iml
+*.ipr
+
+### Eclipse ###
+.apt_generated
+.classpath
+.factorypath
+.project
+.settings
+.springBeans
+.sts4-cache
+
+### NetBeans ###
+/nbproject/private/
+/nbbuild/
+/dist/
+/nbdist/
+/.nb-gradle/
+build/
+!**/src/main/**/build/
+!**/src/test/**/build/
+
+### VS Code ###
+.vscode/
+
+### Mac OS ###
+.DS_Store

README.md

@@ -0,0 +1,173 @@
+# 数据采集 (Data Acquisition)
+
+[![Java](https://img.shields.io/badge/Java-21-orange.svg)](https://www.oracle.com/java/)
+[![Maven](https://img.shields.io/badge/Maven-3.6+-blue.svg)](https://maven.apache.org/)
+[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+从托育机构信息平台 (tuoyu.cpdrc.org.cn) 爬取数据并导出为 Excel 文件的 Java 爬虫工具。
+
+## 快速开始
+
+```bash
+# 编译
+mvn compile
+
+# 运行
+mvn exec:java -Dexec.mainClass=space.anyi.app.Application
+```
+
+运行后会在项目根目录生成 `data.xlsx` 文件。
+
+## 环境要求
+
+| 依赖 | 版本 |
+|------|------|
+| JDK | 21+ |
+| Maven | 3.6+ |
+
+## 运行效果
+
+```
+数据已成功保存到Excel文件: data.xlsx
+共保存 XXX 条记录
+```
+
+## 工作原理
+
+```
+┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
+│  启动程序    │────▶│  请求第1页   │────▶│  获取总页数   │────▶│  遍历所有页  │
+│  (pageSize= │     │  获取总数     │     │  解析响应     │     │  100ms 间隔  │
+│   10,page=1)│     │  解析响应     │     │  决定循环次数  │     │  累积数据     │
+└─────────────┘     └─────────────┘     └─────────────┘     └──────┬──────┘
+                                                                   │
+                                                              ┌────▼─────┐
+                                                              │ 生成 Excel │
+                                                              │ data.xlsx │
+                                                              └──────────┘
+```
+
+## API 数据源
+
+| 项 | 值 |
+|---|---|
+| 目标网站 | https://tuoyu.cpdrc.org.cn/ |
+| API 地址 | `https://tuoyu.cpdrc.org.cn/bapfopm/pub/search/action/queryInfo?pageSize={n}&pageNum={n}` |
+| 请求方式 | GET |
+| 分页策略 | 首次请求获取总页数，然后逐页遍历 |
+| 请求间隔 | 100ms（防止请求过快） |
+
+### API 响应示例
+
+```json
+{
+  "code": 200,
+  "message": "success",
+  "responseData": {
+    "dataList": [
+      {
+        "institution_name": "XX托育中心",
+        "institution_other_name": "XX早教",
+        "institution_type": "全日制",
+        "credit_code": "91XXXXXXXXXXXXXXXX",
+        "address": "XX省XX市XX区XX路XX号",
+        "zoning_name": "XX区",
+        "zoning_code": "110101",
+        "finished_time": "2024-01-01"
+      }
+    ],
+    "totalPage": 50,
+    "totalRecord": 500,
+    "pageSize": 10,
+    "pageIndex": 1,
+    "offset": 0
+  }
+}
+```
+
+## 输出字段
+
+| 列名 | 说明 | JSON 键 |
+|------|------|---------|
+| 机构名称 | 托育机构正式名称 | `institution_name` |
+| 机构别名 | 机构其他名称 | `institution_other_name` |
+| 机构类型 | 机构所属分类 | `institution_type` |
+| 信用代码 | 统一社会信用代码 | `credit_code` |
+| 地址 | 机构详细地址 | `address`（直接匹配） |
+| 所在区域 | 行政区划名称 | `zoning_name` |
+| 区域代码 | 行政区划代码 | `zoning_code` |
+| 完成时间 | 数据录入/完成时间 | `finished_time` |
+
+## 项目结构
+
+```
+src/main/java/space/anyi/app/
+├── Application.java          # 入口：发起请求 → 解析 → 导出 Excel
+└── model/
+    ├── Institution.java      # DTO：机构数据实体，使用 @JsonProperty 映射
+    ├── Response.java         # API 响应信封：code, message, responseData
+    └── ResponseData.java     # 分页数据包装：dataList, totalPage, totalRecord 等
+```
+
+## 数据模型关系
+
+```
+Response
+├── code (int)
+├── message (String)
+└── responseData (ResponseData)
+    ├── dataList (List<Institution>)
+    ├── totalPage (Integer)
+    ├── totalRecord (Integer)
+    ├── pageSize (Integer)
+    ├── pageIndex (Integer)
+    └── offset (Integer)
+```
+
+## 技术栈
+
+| 组件 | 版本 | 用途 |
+|------|------|------|
+| Jackson Databind | 2.21.2 | JSON 反序列化 |
+| Apache POI | 5.5.1 | Excel 文件生成 |
+| Java HttpClient | JDK 内置 | HTTP 请求 |
+
+## 常用命令
+
+| 命令 | 说明 |
+|------|------|
+| `mvn compile` | 编译项目 |
+| `mvn exec:java -Dexec.mainClass=space.anyi.app.Application` | 运行爬虫 |
+| `mvn clean` | 清理构建产物 |
+
+## 注意事项
+
+- **输出文件**: `data.xlsx` 不在 `.gitignore` 中，请勿提交
+- **Excel 格式**: 使用 `HSSFWorkbook` 生成的是旧版 `.xls` 二进制格式，扩展名虽为 `.xlsx`，实际需用 Excel 兼容模式打开
+- **格式升级**: 如需真正的 `.xlsx` 格式，需替换为 `XSSFWorkbook` 并添加 `poi-ooxml` 依赖
+- **执行插件**: `mvn exec:java` 依赖 Maven 默认插件解析，`pom.xml` 中未显式配置 `exec-maven-plugin`
+- **运行时间**: 数据量较大时请耐心等待，程序会自动处理所有页面
+
+## 常见问题
+
+### Q: 运行时报错 "无法解析依赖"？
+
+确保已正确安装 Maven 并能访问中央仓库。可尝试：
+
+```bash
+mvn clean install
+```
+
+### Q: 生成的 Excel 文件打不开？
+
+程序使用 HSSF 格式生成的是 `.xls` 格式文件。用 Excel 打开时可能会提示格式不匹配，点击"是"即可正常打开。如需真正的 `.xlsx` 格式，请参考 [格式升级](#注意事项) 说明。
+
+### Q: 如何调整每页数据量？
+
+修改 `Application.java` 中的 `pageSize` 变量即可：
+
+```java
+int pageSize = 20; // 改为每页20条
+```
+
+注意：过大的 pageSize 可能导致请求超时。

pom.xml

@@ -0,0 +1,39 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+
+    <groupId>space.anyi</groupId>
+    <artifactId>data_acquisition</artifactId>
+    <version>1.0-SNAPSHOT</version>
+    <description>
+        数据采集,
+        https://tuoyu.cpdrc.org.cn/#/
+        https://tuoyu.cpdrc.org.cn/bapfopm/pub/search/action/queryInfo?pageSize=10&amp;pageNum=2
+    </description>
+
+    <properties>
+        <maven.compiler.source>21</maven.compiler.source>
+        <maven.compiler.target>21</maven.compiler.target>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+    </properties>
+
+    <dependencies>
+        <!-- Source: https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind -->
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-databind</artifactId>
+            <version>2.21.2</version>
+            <scope>compile</scope>
+        </dependency>
+        <!-- Source: https://mvnrepository.com/artifact/org.apache.poi/poi-ooxml -->
+        <dependency>
+            <groupId>org.apache.poi</groupId>
+            <artifactId>poi-ooxml</artifactId>
+            <version>5.5.1</version>
+            <scope>compile</scope>
+        </dependency>
+    </dependencies>
+
+</project>

src/main/java/space/anyi/app/Application.java

@@ -0,0 +1,154 @@
+package space.anyi.app;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.poi.ss.usermodel.*;
+import org.apache.poi.xssf.usermodel.XSSFWorkbook;
+import space.anyi.app.model.Institution;
+import space.anyi.app.model.Response;
+
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * @fileName: Application
+ * @projectName: data_acquisition
+ * @package: space.anyi.app
+ * @author: 杨逸
+ * @date:2026/5/1 22:06
+ * @description:
+ */
+public class Application {
+    private  final static String templateUrl = "https://tuoyu.cpdrc.org.cn/bapfopm/pub/search/action/queryInfo?pageSize=%d&pageNum=%d";
+    private final static HttpClient httpClient = HttpClient.newHttpClient();
+    private final static ObjectMapper objectMapper = new ObjectMapper();
+    public static void main(String[] args) {
+        int pageSize = 10;
+        int pageNum = 1;
+        Integer totalPage;
+        //初次获取数据,获取总数据条数
+        String url = templateUrl.formatted(pageSize, pageNum);
+        HttpRequest httpRequest = HttpRequest.newBuilder(URI.create(url)).GET().build();
+        try {
+            //发送请求
+            HttpResponse<String> httpResponse = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
+            //获取响应数据
+            Response response = objectMapper.readValue(httpResponse.body(), Response.class);
+            totalPage = response.getResponseData().getTotalPage();
+            //批量处理
+            handler(totalPage, pageSize);
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        } catch (InterruptedException e) {
+            throw new RuntimeException(e);
+        }
+
+    }
+
+    private static void handler(Integer totalPage, int pageSize) {
+        List<Institution> res = new ArrayList<>();
+        System.out.println("开始爬取数据，共 " + totalPage + " 页...");
+        for (int i = 1; i <= totalPage; i++) {
+            String url = templateUrl.formatted(pageSize, i);
+            HttpRequest httpRequest = HttpRequest.newBuilder(URI.create(url)).GET().build();
+            try {
+                HttpResponse<String> httpResponse = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
+                Response response = objectMapper.readValue(httpResponse.body(), Response.class);
+                //处理数据
+                res.addAll(response.getResponseData().getDataList());
+                //打印进度
+                System.out.printf("\r进度: [%d/%d] 页, 已获取 %d 条记录", i, totalPage, res.size());
+                //休眠1秒
+                //Thread.sleep(100L);
+            } catch (IOException | InterruptedException e) {
+                throw new RuntimeException(e);
+            }
+        }
+        System.out.println(); // 换行
+        saveToExcel(res, "data.xlsx");
+    }
+
+    private static void save(List<Institution> list) {
+        //模拟数据储存
+        for (Institution institution : list) {
+            System.out.println(institution);
+        }
+    }
+
+    private static void saveToExcel(List<Institution> institutions, String filePath) {
+        Workbook workbook = new XSSFWorkbook();
+        Sheet sheet = workbook.createSheet("托育机构信息");
+
+        // 创建标题行样式
+        CellStyle headerStyle = workbook.createCellStyle();
+        Font headerFont = workbook.createFont();
+        headerFont.setBold(true);
+        headerStyle.setFont(headerFont);
+        headerStyle.setFillForegroundColor(IndexedColors.GREY_25_PERCENT.getIndex());
+        headerStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND);
+        headerStyle.setBorderBottom(BorderStyle.THIN);
+        headerStyle.setBorderTop(BorderStyle.THIN);
+        headerStyle.setBorderLeft(BorderStyle.THIN);
+        headerStyle.setBorderRight(BorderStyle.THIN);
+
+        // 创建内容行样式
+        CellStyle contentStyle = workbook.createCellStyle();
+        contentStyle.setBorderBottom(BorderStyle.THIN);
+        contentStyle.setBorderTop(BorderStyle.THIN);
+        contentStyle.setBorderLeft(BorderStyle.THIN);
+        contentStyle.setBorderRight(BorderStyle.THIN);
+
+        // 创建表头
+        String[] headers = {"机构名称", "机构别名", "机构类型", "信用代码", "地址",
+                "所在区域", "区域代码", "完成时间"};
+        Row headerRow = sheet.createRow(0);
+        for (int i = 0; i < headers.length; i++) {
+            Cell cell = headerRow.createCell(i);
+            cell.setCellValue(headers[i]);
+            cell.setCellStyle(headerStyle);
+            sheet.setColumnWidth(i, 5000);
+        }
+
+        // 填充数据
+        int rowNum = 1;
+        for (Institution inst : institutions) {
+            Row row = sheet.createRow(rowNum++);
+
+            row.createCell(0).setCellValue(inst.getInstitutionName());
+            row.createCell(1).setCellValue(inst.getInstitutionOtherName());
+            row.createCell(2).setCellValue(inst.getInstitutionType());
+            row.createCell(3).setCellValue(inst.getCreditCode());
+            row.createCell(4).setCellValue(inst.getAddress());
+            row.createCell(5).setCellValue(inst.getZoningName());
+            row.createCell(6).setCellValue(inst.getZoningCode());
+            row.createCell(7).setCellValue(inst.getFinishedTime());
+
+            // 应用样式
+            for (int i = 0; i < headers.length; i++) {
+                row.getCell(i).setCellStyle(contentStyle);
+            }
+        }
+
+        // 写入文件
+        try (FileOutputStream fileOut = new FileOutputStream(filePath)) {
+            workbook.write(fileOut);
+            System.out.println("\n数据已成功保存到Excel文件: " + filePath);
+            System.out.println("共保存 " + institutions.size() + " 条记录");
+        } catch (IOException e) {
+            System.err.println("保存Excel文件失败: " + e.getMessage());
+            e.printStackTrace();
+        } finally {
+            try {
+                workbook.close();
+            } catch (IOException e) {
+                e.printStackTrace();
+            }
+        }
+    }
+}

src/main/java/space/anyi/app/model/Institution.java

@@ -0,0 +1,115 @@
+package space.anyi.app.model;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * @fileName: Institution
+ * @projectName: data_acquisition
+ * @package: space.anyi.app.model
+ * @author: 杨逸
+ * @date:2026/5/1 22:25
+ * @description: 使用record定义机构实体类
+ */
+public class Institution{
+    //机构id
+    @JsonProperty("zoning_code")
+    private String zoningCode;
+    //通过时间
+    @JsonProperty("finished_time")
+    private String finishedTime;
+    //机构地址
+    private String address;
+    //机构类型
+    @JsonProperty("institution_type")
+    private String institutionType;
+    //地区名称
+    @JsonProperty("zoning_name")
+    private String zoningName;
+    //统一社会信用代码
+    @JsonProperty("credit_code")
+    private String creditCode;
+    //机构简称
+    @JsonProperty("institution_other_name")
+    private String institutionOtherName;
+    //机构注册登记名称
+    @JsonProperty("institution_name")
+    private String institutionName;
+
+    public String getZoningCode() {
+        return zoningCode;
+    }
+
+    public void setZoningCode(String zoningCode) {
+        this.zoningCode = zoningCode;
+    }
+
+    public String getFinishedTime() {
+        return finishedTime;
+    }
+
+    public void setFinishedTime(String finishedTime) {
+        this.finishedTime = finishedTime;
+    }
+
+    public String getAddress() {
+        return address;
+    }
+
+    public void setAddress(String address) {
+        this.address = address;
+    }
+
+    public String getInstitutionType() {
+        return institutionType;
+    }
+
+    public void setInstitutionType(String institutionType) {
+        this.institutionType = institutionType;
+    }
+
+    public String getZoningName() {
+        return zoningName;
+    }
+
+    public void setZoningName(String zoningName) {
+        this.zoningName = zoningName;
+    }
+
+    public String getCreditCode() {
+        return creditCode;
+    }
+
+    public void setCreditCode(String creditCode) {
+        this.creditCode = creditCode;
+    }
+
+    public String getInstitutionOtherName() {
+        return institutionOtherName;
+    }
+
+    public void setInstitutionOtherName(String institutionOtherName) {
+        this.institutionOtherName = institutionOtherName;
+    }
+
+    public String getInstitutionName() {
+        return institutionName;
+    }
+
+    public void setInstitutionName(String institutionName) {
+        this.institutionName = institutionName;
+    }
+
+    @Override
+    public String toString() {
+        return "Institution{" +
+                "zoningCode='" + zoningCode + '\'' +
+                ", finishedTime='" + finishedTime + '\'' +
+                ", address='" + address + '\'' +
+                ", institutionType='" + institutionType + '\'' +
+                ", zoningName='" + zoningName + '\'' +
+                ", creditCode='" + creditCode + '\'' +
+                ", institutionOtherName='" + institutionOtherName + '\'' +
+                ", institutionName='" + institutionName + '\'' +
+                '}';
+    }
+}

src/main/java/space/anyi/app/model/Response.java

@@ -0,0 +1,39 @@
+package space.anyi.app.model;
+
+/**
+ * @fileName: Response
+ * @projectName: data_acquisition
+ * @package: space.anyi.app.model
+ * @author: 杨逸
+ * @date:2026/5/1 22:33
+ * @description: TODO
+ */
+public class Response {
+    private int code;
+    private String message;
+    private ResponseData responseData;
+
+    public int getCode() {
+        return code;
+    }
+
+    public void setCode(int code) {
+        this.code = code;
+    }
+
+    public String getMessage() {
+        return message;
+    }
+
+    public void setMessage(String message) {
+        this.message = message;
+    }
+
+    public ResponseData getResponseData() {
+        return responseData;
+    }
+
+    public void setResponseData(ResponseData responseData) {
+        this.responseData = responseData;
+    }
+}

src/main/java/space/anyi/app/model/ResponseData.java

@@ -0,0 +1,68 @@
+package space.anyi.app.model;
+
+import java.util.List;
+
+/**
+ * @fileName: ResponseData
+ * @projectName: data_acquisition
+ * @package: space.anyi.app.model
+ * @author: 杨逸
+ * @date:2026/5/1 22:34
+ * @description:
+ */
+public class ResponseData {
+    private List<Institution> dataList;
+    private Integer offset;
+    private Integer pageIndex;
+    private Integer pageSize;
+    private Integer totalPage;
+    private Integer totalRecord;
+
+    public List<Institution> getDataList() {
+        return dataList;
+    }
+
+    public void setDataList(List<Institution> dataList) {
+        this.dataList = dataList;
+    }
+
+    public Integer getOffset() {
+        return offset;
+    }
+
+    public void setOffset(Integer offset) {
+        this.offset = offset;
+    }
+
+    public Integer getPageIndex() {
+        return pageIndex;
+    }
+
+    public void setPageIndex(Integer pageIndex) {
+        this.pageIndex = pageIndex;
+    }
+
+    public Integer getPageSize() {
+        return pageSize;
+    }
+
+    public void setPageSize(Integer pageSize) {
+        this.pageSize = pageSize;
+    }
+
+    public Integer getTotalPage() {
+        return totalPage;
+    }
+
+    public void setTotalPage(Integer totalPage) {
+        this.totalPage = totalPage;
+    }
+
+    public Integer getTotalRecord() {
+        return totalRecord;
+    }
+
+    public void setTotalRecord(Integer totalRecord) {
+        this.totalRecord = totalRecord;
+    }
+}