Compare commits

...

55 Commits

Author SHA1 Message Date
Jochen Topf 30db69822d Github action: Modernize/fix CI 2026-07-26 16:49:46 +02:00
Jochen Topf 0265ab5f3d Release version 1.7.0 2026-07-26 16:42:30 +02:00
Jochen Topf 5b5ae13a4a Move osmpbf-outline tool from "tools" into "demo" directory
This isn't really a useful tool for normal users. It is a demo/example
program for developers to see how the library can be used.
2026-07-26 16:38:45 +02:00
Jochen Topf c33d74be06 Avoid global variables 2026-07-24 15:53:49 +02:00
Jochen Topf 01f1a3f0be Detect lz4 or zstd compressed data 2026-07-24 15:53:49 +02:00
Jochen Topf 719a26d4af Various code modernizations/fixes 2026-07-24 15:53:45 +02:00
Jochen Topf 5b6d2afa9c Use C++ versions of headers 2026-07-24 14:53:28 +02:00
Jochen Topf 830df177b5 Mark err() function as noreturn 2026-07-24 14:52:20 +02:00
Jochen Topf eda1580ce9 CMake config: Enable generation of compile_commands.json 2026-07-24 14:51:49 +02:00
Jochen Topf 9eddf3d1f8
Merge pull request #95 from simon04/fix-94
Fix BSS buffer overflow in osmpbf-outline zlib decompression
2026-07-24 14:51:08 +02:00
Simon Legner 8857d08d80 Fix BSS buffer overflow in osmpbf-outline zlib decompression
Fixes #94.
2026-07-22 22:16:52 +02:00
Leonard Ehrenfried 28f76c6bf0
Merge pull request #93 from sebastic/java-version
Set Java version to 1.8 to match Maven build.
2026-03-11 10:21:39 +01:00
Bas Couwenberg 5bb6bbfc62 Set Java version to 1.8 to match Maven build. 2026-03-10 19:38:12 +01:00
Leonard Ehrenfried bc457ce7c0
Bump version to 1.6.1 2025-12-20 10:45:14 +01:00
Leonard Ehrenfried 7c3f68f622
Merge pull request #92 from openstreetmap/upgrade-protobuf
Upgrade Java's protobuf to 4.33.2
2025-12-20 10:31:55 +01:00
Leonard Ehrenfried ca046bcff4
Add badges for Java 2025-12-20 09:51:03 +01:00
Leonard Ehrenfried 4048bf6e35
Add changelog 2025-12-20 09:46:06 +01:00
Leonard Ehrenfried de424ed053
Upgrade protobuf to 4.33.2 2025-12-20 09:35:29 +01:00
Jochen Topf 0cc6f10fcf CI: Fix macOS build 2025-12-03 09:20:32 +01:00
Jochen Topf 32c3e92166 Update CI to run on Ubuntu 22.04 and 24.04 2025-06-11 11:07:09 +02:00
Jochen Topf a2e364e5e7
Merge pull request #87 from leonardehrenfried/master
Release version 1.6.0 of Java
2024-12-10 09:55:59 +01:00
Leonard Ehrenfried 92a9c8d572
Don't wait for publishing 2024-12-10 08:48:05 +01:00
Leonard Ehrenfried 68d2b397a1
Configure auto-publish 2024-12-10 08:20:51 +01:00
Leonard Ehrenfried c5044e0f0a
Release version 1.6.0 2024-12-10 08:20:09 +01:00
Jochen Topf a8c4dd1c11
Merge pull request #84 from leonardehrenfried/master
Update to latest version of protobuf compiler and Java runtime
2024-11-13 11:09:54 +01:00
Leonard Ehrenfried 505052800e
Use single variable for compiler and runtime 2024-11-07 22:09:23 +01:00
Leonard Ehrenfried f427d7458e
Update to latest version of protobuf compiler and Java runtime 2024-11-07 22:06:32 +01:00
Jochen Topf 7050448620
Merge pull request #83 from codesoap/master
Remove trailing newlines from proto files
2024-09-29 10:15:27 +02:00
codesoap ecf26bf6a3 Remove trailing newlines from proto files 2024-09-29 07:35:19 +02:00
Jochen Topf 65e7e976f5
Merge pull request #79 from simon04/patch-5
actions/setup-java@v4
2024-03-28 10:56:52 +01:00
Simon Legner fa266b6f9b actions/setup-java@v4 2024-03-27 22:13:01 +01:00
Jochen Topf db10ff0949 Modernize github actions 2024-03-11 09:26:22 +01:00
Jochen Topf 802d4603b2 Remove Makefiles (have been deprecated a while) 2024-03-11 09:19:08 +01:00
Jochen Topf 30097cd4cb CMake config: Set minimum C++ version needed in a different way
This setting should be overwritten if CMake find that some library needs
an even more modern version.
2024-01-07 17:41:31 +01:00
Jochen Topf 4e009b0055
Merge pull request #77 from simonpoole/expose_CompressFlags
Extract and make CompressFlags public
2024-01-07 14:03:33 +01:00
simonpoole 8219186b2e Extract and make CompressFlags public
Fixes https://github.com/openstreetmap/OSM-binary/issues/68
2024-01-07 11:44:09 +01:00
Jochen Topf 374895c1e3 Set C++11 as minimum supported C++ version
Fixes #71
2023-10-02 10:04:01 +02:00
Jochen Topf 3b3b9cd2cd
Merge pull request #70 from openstreetmap/dependabot/maven/com.google.protobuf-protobuf-java-3.16.3
Bump protobuf-java from 3.16.1 to 3.16.3
2023-10-02 09:42:27 +02:00
Jochen Topf 7259487654 Github action: Use checkout v4 2023-10-02 09:37:52 +02:00
Jochen Topf 85e763ea0f Github action: Use clang 15 on Ubuntu 22.04 2023-10-02 09:37:52 +02:00
Jochen Topf ffd833fb22
Merge pull request #73 from probonopd/patch-1
#include <unistd.h>
2023-10-02 09:37:29 +02:00
probonopd 6483ae0dcb
#include <unistd.h>
Fixes compilation on FreeBSD
Closes #72
2023-10-01 23:58:09 +02:00
dependabot[bot] 587cbd2776
Bump protobuf-java from 3.16.1 to 3.16.3
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.16.1 to 3.16.3.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v3.16.1...v3.16.3)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-04 23:48:44 +00:00
Jochen Topf 37adab91e5 Remove symlink from osmpbf to src, use osmpbf directly 2022-05-27 12:36:32 +02:00
Jochen Topf 4195fe6557 CMake config: Remove unused line 2022-05-27 12:35:46 +02:00
Jochen Topf 6e9c00769f CMake: Do not set C++ version any more.
At least C++11 is standard in modern compilers anyway. Only macOS seems
to need an explicit set for this.
2022-05-27 11:48:42 +02:00
Jochen Topf cffbe661b1 Github Actions: Build on Ubuntu 20.04/22.04, not 18.04 any more 2022-05-27 11:06:51 +02:00
Jochen Topf a3f9e58622 Show packaging status in README.md 2022-01-08 10:07:10 +01:00
Jochen Topf 278c07098f Guthub action: Remove Ubuntu 16 from CI build
Because thas system isn't supported any more by github.
2022-01-08 10:06:00 +01:00
Jochen Topf edad45f7ab
Merge pull request #67 from openstreetmap/dependabot/maven/com.google.protobuf-protobuf-java-3.16.1
Bump protobuf-java from 3.13.0 to 3.16.1
2022-01-08 10:04:02 +01:00
dependabot[bot] 6d91b131e1
Bump protobuf-java from 3.13.0 to 3.16.1
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.13.0 to 3.16.1.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/master/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v3.13.0...v3.16.1)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-01-07 22:36:35 +00:00
Jochen Topf 7e111e6791
Merge pull request #66 from openstreetmap/ant-build
Ant build
2021-01-09 21:49:01 +01:00
Jochen Topf 0e99a6fef2 Document build with Ant 2021-01-09 10:14:23 +01:00
Jochen Topf d9ee34140d Revert "Delete obsolete build.sh"
This reverts commit 33e940238b.

Ant build is still used by Debian
2021-01-09 09:54:12 +01:00
Jochen Topf 40d041e344 Revert "Delete obsolete Ant build file"
This reverts commit fa86a8a640.

Ant build is still used by Debian
2021-01-09 09:47:02 +01:00
23 changed files with 329 additions and 215 deletions

View File

@ -19,6 +19,6 @@ runs:
working-directory: build
- name: Run outline
run: build/tools/osmpbf-outline resources/sample.pbf
run: build/demo/osmpbf-outline resources/sample.pbf
shell: bash

View File

@ -3,56 +3,56 @@ name: C CI
on: [push, pull_request]
jobs:
ubuntu16:
runs-on: ubuntu-16.04
ubuntu22:
runs-on: ubuntu-22.04
env:
CC: gcc
CXX: g++
CXXFLAGS: -Werror -Wall -pedantic
BUILD_TYPE: Debug
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-ubuntu
- uses: ./.github/actions/build
ubuntu18:
runs-on: ubuntu-18.04
ubuntu24:
runs-on: ubuntu-24.04
env:
CC: gcc
CXX: g++
CXXFLAGS: -Werror -Wall -pedantic
BUILD_TYPE: Debug
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-ubuntu
- uses: ./.github/actions/build
ubuntu20:
runs-on: ubuntu-20.04
ubuntu26:
runs-on: ubuntu-26.04
env:
CC: gcc
CXX: g++
CXXFLAGS: -Werror -Wall -pedantic
BUILD_TYPE: Debug
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-ubuntu
- uses: ./.github/actions/build
ubuntu20-clang:
runs-on: ubuntu-20.04
ubuntu24-clang:
runs-on: ubuntu-24.04
env:
CC: clang-10
CXX: clang++-10
CC: clang-18
CXX: clang++-18
CXXFLAGS: -Werror -Wall -pedantic
BUILD_TYPE: Debug
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-ubuntu
- uses: ./.github/actions/build
ubuntu20-release:
runs-on: ubuntu-20.04
ubuntu24-release:
runs-on: ubuntu-24.04
env:
CC: gcc
CXX: g++
@ -60,37 +60,26 @@ jobs:
CXXFLAGS: -Werror -Wall -pedantic -Wno-array-bounds
BUILD_TYPE: Release
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-ubuntu
- uses: ./.github/actions/build
ubuntu20-make:
runs-on: ubuntu-20.04
env:
CC: gcc
CXX: g++
CXXFLAGS: -Werror -Wall -pedantic
steps:
- uses: actions/checkout@v2
- uses: ./.github/actions/install-ubuntu
- run: make -C src
macos:
runs-on: macos-latest
env:
CC: clang
CXX: clang++
CXXFLAGS: -Werror -Wall -pedantic
CXXFLAGS: -Werror -Wall -pedantic -Wno-nullability-extension
BUILD_TYPE: Debug
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-macos
- uses: ./.github/actions/build
windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- uses: ./.github/actions/install-windows
- uses: ./.github/actions/build-windows

View File

@ -3,15 +3,81 @@ name: Java CI
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
maven-ubuntu22:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v7
- run: sudo apt-get install protobuf-compiler
- name: Set up JDK 1.8
uses: actions/setup-java@v1
- name: Set up JDK 8
uses: actions/setup-java@v5
with:
java-version: 1.8
distribution: zulu
java-version: 8
- name: Build with Maven
run: mvn -B package --file pom.xml
maven-ubuntu24:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
- run: sudo apt-get install protobuf-compiler
- name: Set up JDK 8
uses: actions/setup-java@v5
with:
distribution: zulu
java-version: 8
- name: Build with Maven
run: mvn -B package --file pom.xml
maven-ubuntu26:
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7
- run: sudo apt-get install protobuf-compiler
- name: Set up JDK 8
uses: actions/setup-java@v5
with:
distribution: zulu
java-version: 8
- name: Build with Maven
run: mvn -B package --file pom.xml
ant-ubuntu22:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v7
- run: sudo apt-get install protobuf-compiler libprotobuf-java
- name: Set up JDK 8
uses: actions/setup-java@v5
with:
distribution: zulu
java-version: 8
- name: Build with Ant
run: ant
ant-ubuntu24:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
- run: sudo apt-get install protobuf-compiler libprotobuf-java
- name: Set up JDK 8
uses: actions/setup-java@v5
with:
distribution: zulu
java-version: 8
- name: Build with Ant
run: ant
ant-ubuntu26:
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7
- run: sudo apt-get install protobuf-compiler libprotobuf-java
- name: Set up JDK 8
uses: actions/setup-java@v5
with:
distribution: zulu
java-version: 8
- name: Build with Ant
run: ant

View File

@ -1,5 +1,21 @@
## Unreleased
## Release notes for 1.7.0 (2026-07-26)
- C++: Fix buffer overflow in osmpbf-outline tool [#95](https://github.com/openstreetmap/OSM-binary/pull/95)
- C++: Various small code cleanups and modernizations
- C++: osmpbf-outline tool moved to demo directory, it isn't installed by default any more. This tool was never really useful for normal users, it is more a demo/example showing how to use the library
Note to packagers: You should remove the osmpbf-outline tool from packages.
## Release notes for 1.6.1 (2025-12-20)
- Java: Update to latest protoc and protobuf runtime versions [#92](https://github.com/openstreetmap/OSM-binary/pull/92)
## Release notes for 1.6.0 (2024-12-10)
- Java: Update to latest protoc and protobuf runtime versions [#84](https://github.com/openstreetmap/OSM-binary/pull/84)
## Release notes for 1.5.0 (2021-01-06)
- C: Support hardening buildflags set in the environment ([09845ca4](https://github.com/openstreetmap/OSM-binary/commit/09845ca4087c7404b1de33914233dbf53f9de4c3))

View File

@ -1,10 +1,18 @@
cmake_minimum_required(VERSION 3.7)
cmake_minimum_required(VERSION 3.10)
project(osmpbf VERSION 1.5.0)
project(osmpbf VERSION 1.7.0)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
include(GNUInstallDirs)
set(CMAKE_CXX_STANDARD 11)
# This is needed for the protobuf_generate_cpp() function to work on newer
# versions of the Protobuf CMake config.
set(protobuf_MODULE_COMPATIBLE ON CACHE BOOL "")
# This is needed so that we pick up the (more modern) CONFIG mode cmake file
# before the (older) MODULE mode cmake config file. Seems to be needed on macOS.
set(CMAKE_FIND_PACKAGE_PREFER_CONFIG TRUE)
find_package(Protobuf REQUIRED)
@ -12,5 +20,5 @@ add_subdirectory(osmpbf)
# The osmpbf-outline tool does not compile on Windows because getopt.h is missing
if(NOT MSVC)
add_subdirectory(tools)
add_subdirectory(demo)
endif()

View File

@ -3,7 +3,7 @@
https://github.com/openstreetmap/OSM-binary
Osmpbf is a Java/C library to read and write OpenStreetMap PBF files.
Osmpbf is a Java/C++ library to read and write OpenStreetMap PBF files.
PBF (Protocol buffer Binary Format) is a binary file format for OpenStreetMap
data that uses Google Protocol Buffers as low-level storage.
@ -13,35 +13,49 @@ Note that this is a low-level library that does only part of the
encoding/decoding needed for actually writing/reading an OSM PBF file. For
something more complete see [libosmium](https://osmcode.org/libosmium/).
[![Github Actions C Build Status](https://github.com/openstreetmap/OSM-binary/workflows/C%20CI/badge.svg?branch=master)](https://github.com/openstreetmap/OSM-binary/actions)
[![Github Actions Java Build Status](https://github.com/openstreetmap/OSM-binary/workflows/Java%20CI/badge.svg?branch=master)](https://github.com/openstreetmap/OSM-binary/actions)
[![Github Actions C++ CI](https://github.com/openstreetmap/OSM-binary/actions/workflows/c.yml/badge.svg)](https://github.com/openstreetmap/OSM-binary/actions/workflows/c.yml)
[![Github Actions Java CI](https://github.com/openstreetmap/OSM-binary/actions/workflows/java.yml/badge.svg)](https://github.com/openstreetmap/OSM-binary/actions/workflows/java.yml)
[![Packaging status](https://repology.org/badge/tiny-repos/libosmpbf.svg)](https://repology.org/project/libosmpbf/versions)
## Java Version
We publish the Java library to [Maven Central](https://search.maven.org/):
### Building with Maven
We publish the Java library to [Maven Central](https://mvnrepository.com/artifact/org.openstreetmap.pbf/osmpbf): ![Maven Central](https://img.shields.io/maven-central/v/org.openstreetmap.pbf/osmpbf.svg)
```xml
<dependency>
<groupId>org.openstreetmap.pbf</groupId>
<artifactId>osmpbf</artifactId>
<version>1.5.0</version>
<version>${LATEST_VERSION}</version>
</dependency>
```
To build the Java library run:
```sh
mvn package
```
For a Java usage example, see [`ReadFileTest`](https://github.com/openstreetmap/OSM-binary/blob/master/test.java/crosby/binary/ReadFileTest.java)
For a Java usage example, see
[`ReadFileTest`](https://github.com/openstreetmap/OSM-binary/blob/master/test.java/crosby/binary/ReadFileTest.java).
### Building with Ant
If you can not use Maven for some reason you can use the
[Ant](https://ant.apache.org/) instead:
```sh
ant
```
This will build `osmpbf.jar` in the main directory.
This build is also used for Debian packaging.
## C++ Version
(Earlier versions used Makefiles for building. Please switch to the CMake-based
build, the Makefiles are deprecated and will be removed in a future version.)
To compile:
```sh
@ -56,11 +70,11 @@ To install:
make install
```
There is a tool named osmpbf-outline that shows a debug output of the contents
of a PBF file. To run it:
There is a demo tool named osmpbf-outline that shows a debug output of some of
the contents of a PBF file. To run it:
```sh
tools/osmpbf-outline osm-file.osm.pbf
demo/osmpbf-outline osm-file.osm.pbf
```

2
build.sh Normal file
View File

@ -0,0 +1,2 @@
protoc --java_out=generated.java osmpbf/osmformat.proto
protoc --java_out=generated.java osmpbf/fileformat.proto

42
build.xml Normal file
View File

@ -0,0 +1,42 @@
<?xml version="1.0" encoding="utf-8" ?>
<project name="osmpbf" default="all" basedir=".">
<property name="src.dir" location="src.java"/>
<property name="src.generated.dir" location="generated.java"/>
<property name="build.dir" location="build"/>
<property name="build.jar" location="osmpbf.jar"/>
<target name="init">
<mkdir dir="${src.generated.dir}"/>
<mkdir dir="${build.dir}"/>
</target>
<target name="clean">
<delete dir="${src.generated.dir}"/>
<delete dir="${build.dir}"/>
<delete file="${build.jar}"/>
</target>
<target name="build" depends="init">
<exec executable="/bin/sh">
<arg value="./build.sh"/>
</exec>
<javac includeantruntime="false" destdir="${build.dir}" source="1.8" target="1.8">
<src path="${src.dir}"/>
<src path="${src.generated.dir}"/>
<classpath>
<fileset file="/usr/share/java/protobuf.jar"/>
<fileset file="/usr/share/java/protobuf-java.jar"/>
</classpath>
</javac>
</target>
<target name="dist" depends="build">
<jar destfile="${build.jar}" basedir="${build.dir}"/>
</target>
<target name="all" depends="dist">
</target>
</project>

View File

View File

@ -10,6 +10,7 @@ add_executable(osmpbf-outline osmpbf-outline.cpp)
target_include_directories(osmpbf-outline SYSTEM PRIVATE ${ZLIB_INCLUDE_DIR})
target_link_libraries(osmpbf-outline PRIVATE osmpbf ZLIB::ZLIB protobuf::libprotobuf)
set_property(TARGET osmpbf-outline PROPERTY CXX_STANDARD 11)
install(TARGETS osmpbf-outline RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
install(FILES osmpbf-outline.1 DESTINATION ${CMAKE_INSTALL_MANDIR}/man1)
#install(TARGETS osmpbf-outline RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
#install(FILES osmpbf-outline.1 DESTINATION ${CMAKE_INSTALL_MANDIR}/man1)

View File

@ -1,8 +1,11 @@
// used for 'isatty'
#include <unistd.h>
// used for va_list in debug-print methods
#include <stdarg.h>
#include <cstdarg>
// file io lib
#include <stdio.h>
#include <cstdio>
// getopt is used to check for the --color-flag
#include <getopt.h>
@ -20,47 +23,31 @@
// this is the header to pbf format
#include <osmpbf/osmpbf.h>
namespace {
// should the output use color?
bool usecolor = false;
// buffer for reading a compressed blob from file
char buffer[OSMPBF::max_uncompressed_blob_size];
// buffer for decompressing the blob
char unpack_buffer[OSMPBF::max_uncompressed_blob_size];
// pbf struct of a BlobHeader
OSMPBF::BlobHeader blobheader;
// pbf struct of a Blob
OSMPBF::Blob blob;
// pbf struct of an OSM HeaderBlock
OSMPBF::HeaderBlock headerblock;
// pbf struct of an OSM PrimitiveBlock
OSMPBF::PrimitiveBlock primblock;
// prints a formatted message to stdout, optionally color coded
void msg(const char* format, int color, va_list args) {
if (usecolor) {
fprintf(stdout, "\x1b[0;%dm", color);
std::fprintf(stdout, "\x1b[0;%dm", color);
}
vfprintf(stdout, format, args);
std::vfprintf(stdout, format, args);
if (usecolor) {
fprintf(stdout, "\x1b[0m\n");
std::fprintf(stdout, "\x1b[0m\n");
} else {
fprintf(stdout, "\n");
std::fprintf(stdout, "\n");
}
}
// prints a formatted message to stdout, color coded to red
void err(const char* format, ...) {
[[noreturn]] void err(const char* format, ...) {
va_list args;
va_start(args, format);
msg(format, 31, args);
va_end(args);
exit(1);
std::exit(1);
}
// prints a formatted message to stdout, color coded to yellow
@ -87,23 +74,25 @@ void debug(const char* format, ...) {
va_end(args);
}
} // anonymous namespace
// application main method
int main(int argc, char *argv[]) {
// check if the output is a tty so we can use colors
#ifdef WIN32
usecolor = 0;
usecolor = false;
#else
usecolor = isatty(1);
#endif
static struct option long_options[] = {
{"color", no_argument, 0, 'c'},
{0, 0, 0, 0}
option const long_options[] = {
{"color", no_argument, nullptr, 'c'},
{nullptr, 0, nullptr, 0}
};
while (1) {
int c = getopt_long(argc, argv, "c", long_options, 0);
while (true) {
int const c = getopt_long(argc, argv, "c", long_options, nullptr);
if (c == -1) {
break;
@ -114,7 +103,7 @@ int main(int argc, char *argv[]) {
usecolor = true;
break;
default:
exit(1);
return 1;
}
}
@ -124,19 +113,25 @@ int main(int argc, char *argv[]) {
}
// open specified file
FILE *fp = fopen(argv[optind], "rb");
FILE *fp = std::fopen(argv[optind], "rb");
if (!fp) {
err("can't open file '%s'", argv[optind]);
}
// buffer for reading a compressed blob from file
static char buffer[OSMPBF::max_uncompressed_blob_size];
// buffer for decompressing the blob
static char unpack_buffer[OSMPBF::max_uncompressed_blob_size];
// read while the file has not reached its end
while (!feof(fp)) {
while (!std::feof(fp)) {
// storage of size, used multiple times
int32_t sz;
std::uint32_t sz = 0;
// read the first 4 bytes of the file, this is the size of the blob-header
if (fread(&sz, sizeof(sz), 1, fp) != 1) {
if (std::fread(&sz, sizeof(sz), 1, fp) != 1) {
break; // end of file reached
}
@ -149,12 +144,15 @@ int main(int argc, char *argv[]) {
}
// read the blob-header from the file
if (fread(buffer, sz, 1, fp) != 1) {
if (std::fread(buffer, sz, 1, fp) != 1) {
err("unable to read blob-header from file");
}
// pbf struct of a BlobHeader
OSMPBF::BlobHeader blobheader;
// parse the blob-header from the read-buffer
if (!blobheader.ParseFromArray(buffer, sz)) {
if (!blobheader.ParseFromArray(buffer, static_cast<int>(sz))) {
err("unable to parse blob header");
}
@ -177,12 +175,15 @@ int main(int argc, char *argv[]) {
}
// read the blob from the file
if (fread(buffer, sz, 1, fp) != 1) {
if (std::fread(buffer, sz, 1, fp) != 1) {
err("unable to read blob from file");
}
// pbf struct of a Blob
OSMPBF::Blob blob;
// parse the blob from the read-buffer
if (!blob.ParseFromArray(buffer, sz)) {
if (!blob.ParseFromArray(buffer, static_cast<int>(sz))) {
err("unable to parse blob");
}
@ -201,15 +202,15 @@ int main(int argc, char *argv[]) {
sz = blob.raw().size();
// check that raw_size is set correctly
if (sz != blob.raw_size()) {
warn(" reports wrong raw_size: %u bytes", blob.raw_size());
if (sz != static_cast<std::uint32_t>(blob.raw_size())) {
warn(" reports wrong raw_size: %d bytes", blob.raw_size());
}
// tell about the blob-data
debug(" contains uncompressed data: %u bytes", sz);
// copy the uncompressed data over to the unpack_buffer
memcpy(unpack_buffer, buffer, sz);
std::memcpy(unpack_buffer, buffer, sz);
}
// if the blob has zlib-compressed data
@ -229,6 +230,12 @@ int main(int argc, char *argv[]) {
debug(" contains zlib-compressed data: %u bytes", sz);
debug(" uncompressed size: %u bytes", blob.raw_size());
// ensure the raw_size fits into the unpack_buffer, otherwise
// zlib would inflate past the end of the fixed-size buffer
if (blob.raw_size() < 0 || blob.raw_size() > OSMPBF::max_uncompressed_blob_size) {
err(" raw_size is bigger then allowed (%d > %u)", blob.raw_size(), OSMPBF::max_uncompressed_blob_size);
}
// zlib information
z_stream z;
@ -270,9 +277,6 @@ int main(int argc, char *argv[]) {
warn(" contains several data streams");
}
// we have at least one datastream
found_data = true;
// tell about the compressed data
debug(" contains lzma-compressed data: %u bytes", blob.lzma_data().size());
debug(" uncompressed size: %u bytes", blob.raw_size());
@ -281,6 +285,36 @@ int main(int argc, char *argv[]) {
err(" lzma-decompression is not supported");
}
// if the blob has lz4-compressed data
if (blob.has_lz4_data()) {
// issue a warning if there is more than one data steam, a blob may only contain one data stream
if (found_data) {
warn(" contains several data streams");
}
// tell about the compressed data
debug(" contains lz4-compressed data: %u bytes", blob.lz4_data().size());
debug(" uncompressed size: %u bytes", blob.raw_size());
// issue a warning, lz4 compression is not yet supported
err(" lz4-decompression is not supported");
}
// if the blob has zstd-compressed data
if (blob.has_zstd_data()) {
// issue a warning if there is more than one data steam, a blob may only contain one data stream
if (found_data) {
warn(" contains several data streams");
}
// tell about the compressed data
debug(" contains zstd-compressed data: %u bytes", blob.zstd_data().size());
debug(" uncompressed size: %u bytes", blob.raw_size());
// issue a warning, zstd compression is not yet supported
err(" zstd-decompression is not supported");
}
// check we have at least one data-stream
if (!found_data) {
err(" does not contain any known data stream");
@ -291,28 +325,31 @@ int main(int argc, char *argv[]) {
// tell about the OSMHeader blob
info(" OSMHeader");
// pbf struct of an OSM HeaderBlock
OSMPBF::HeaderBlock headerblock;
// parse the HeaderBlock from the blob
if (!headerblock.ParseFromArray(unpack_buffer, sz)) {
if (!headerblock.ParseFromArray(unpack_buffer, static_cast<int>(sz))) {
err("unable to parse header block");
}
// tell about the bbox
if (headerblock.has_bbox()) {
OSMPBF::HeaderBBox bbox = headerblock.bbox();
OSMPBF::HeaderBBox const& bbox = headerblock.bbox();
debug(" bbox: %.7f,%.7f,%.7f,%.7f",
(double)bbox.left() / OSMPBF::lonlat_resolution,
(double)bbox.bottom() / OSMPBF::lonlat_resolution,
(double)bbox.right() / OSMPBF::lonlat_resolution,
(double)bbox.top() / OSMPBF::lonlat_resolution);
static_cast<double>(bbox.left()) / OSMPBF::lonlat_resolution,
static_cast<double>(bbox.bottom()) / OSMPBF::lonlat_resolution,
static_cast<double>(bbox.right()) / OSMPBF::lonlat_resolution,
static_cast<double>(bbox.top()) / OSMPBF::lonlat_resolution);
}
// tell about the required features
for (int i = 0, l = headerblock.required_features_size(); i < l; i++) {
for (int i = 0, l = headerblock.required_features_size(); i < l; ++i) {
debug(" required_feature: %s", headerblock.required_features(i).c_str());
}
// tell about the optional features
for (int i = 0, l = headerblock.optional_features_size(); i < l; i++) {
for (int i = 0, l = headerblock.optional_features_size(); i < l; ++i) {
debug(" optional_feature: %s", headerblock.optional_features(i).c_str());
}
@ -329,8 +366,11 @@ int main(int argc, char *argv[]) {
// tell about the OSMData blob
info(" OSMData");
// pbf struct of an OSM PrimitiveBlock
OSMPBF::PrimitiveBlock primblock;
// parse the PrimitiveBlock from the blob
if (!primblock.ParseFromArray(unpack_buffer, sz)) {
if (!primblock.ParseFromArray(unpack_buffer, static_cast<int>(sz))) {
err("unable to parse primitive block");
}
@ -347,11 +387,11 @@ int main(int argc, char *argv[]) {
debug(" primitivegroups: %u groups", primblock.primitivegroup_size());
// iterate over all PrimitiveGroups
for (int i = 0, l = primblock.primitivegroup_size(); i < l; i++) {
for (int i = 0, l = primblock.primitivegroup_size(); i < l; ++i) {
// one PrimitiveGroup from the the Block
OSMPBF::PrimitiveGroup pg = primblock.primitivegroup(i);
OSMPBF::PrimitiveGroup const& pg = primblock.primitivegroup(i);
bool found_items=false;
bool found_items = false;
// tell about nodes
if (pg.nodes_size() > 0) {
@ -406,7 +446,7 @@ int main(int argc, char *argv[]) {
}
// close the file pointer
fclose(fp);
std::fclose(fp);
// clean up the protobuf lib
google::protobuf::ShutdownProtobufLibrary();

View File

@ -18,7 +18,7 @@
// this describes the high-level OSM objects
#include <osmpbf/osmformat.pb.h> // IWYU pragma: export
#define OSMPBF_VERSION "1.5.0"
#define OSMPBF_VERSION "1.7.0"
namespace OSMPBF {

1
osmpbf
View File

@ -1 +0,0 @@
src

View File

@ -1,10 +1,13 @@
protobuf_generate_cpp(CPPS HS fileformat.proto osmformat.proto)
add_library(osmpbf STATIC ${CPPS})
target_compile_features(osmpbf PUBLIC cxx_std_11)
target_link_libraries(osmpbf PRIVATE protobuf::libprotobuf)
target_include_directories(osmpbf SYSTEM PUBLIC ${Protobuf_INCLUDE_DIRS})
install(TARGETS osmpbf ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
add_library(osmpbf_shared SHARED ${CPPS})
target_compile_features(osmpbf_shared PUBLIC cxx_std_11)
target_link_libraries(osmpbf_shared PRIVATE protobuf::libprotobuf)
target_include_directories(osmpbf_shared SYSTEM PUBLIC ${Protobuf_INCLUDE_DIRS})
set_target_properties(osmpbf_shared PROPERTIES OUTPUT_NAME osmpbf

View File

@ -65,5 +65,3 @@ message BlobHeader {
optional bytes indexdata = 2;
required int32 datasize = 3;
}

View File

@ -260,4 +260,3 @@ message Relation {
repeated sint64 memids = 9 [packed = true]; // DELTA encoded
repeated MemberType types = 10 [packed = true];
}

47
pom.xml
View File

@ -6,7 +6,7 @@
<groupId>org.openstreetmap.pbf</groupId>
<artifactId>osmpbf</artifactId>
<packaging>jar</packaging>
<version>1.5.0</version>
<version>1.7.0</version>
<name>OSM-Binary</name>
<description>Library for the OpenStreetMap PBF format</description>
<url>https://github.com/openstreetmap/OSM-binary</url>
@ -31,9 +31,20 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<protobuf.version>4.33.2</protobuf.version>
</properties>
<build>
<extensions>
<!-- makes build variables like os.detected.classifier available.
use to select the correct protoc instance for the OS.
see protobuf-maven-plugin below -->
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.7.1</version>
</extension>
</extensions>
<sourceDirectory>${basedir}/src.java</sourceDirectory>
<testSourceDirectory>${basedir}/test.java</testSourceDirectory>
@ -59,7 +70,7 @@
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.6.1</version>
<configuration>
<protocExecutable>protoc</protocExecutable>
<protocArtifact>com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}</protocArtifact>
</configuration>
<executions>
<execution>
@ -70,7 +81,7 @@
</goals>
<phase>generate-sources</phase>
<configuration>
<protoSourceRoot>${basedir}/src</protoSourceRoot>
<protoSourceRoot>${basedir}/osmpbf</protoSourceRoot>
<includes>
<param>**/*.proto</param>
</includes>
@ -122,16 +133,19 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.sonatype.central</groupId>
<artifactId>central-publishing-maven-plugin</artifactId>
<version>0.6.0</version>
<extensions>true</extensions>
<configuration>
<publishingServerId>central</publishingServerId>
<autoPublish>true</autoPublish>
</configuration>
</plugin>
</plugins>
</build>
<pluginRepositories>
<pluginRepository>
<id>dtrott</id>
<url>http://maven.davidtrott.com/repository</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>junit</groupId>
@ -143,19 +157,8 @@
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.13.0</version>
<version>${protobuf.version}</version>
</dependency>
</dependencies>
<distributionManagement>
<repository>
<id>ossrh</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2</url>
</repository>
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
</project>

View File

@ -25,10 +25,6 @@ import java.io.OutputStream;
import java.util.ArrayList;
import java.util.List;
enum CompressFlags {
NONE, DEFLATE
}
public class BlockOutputStream implements Closeable, Flushable {
public BlockOutputStream(OutputStream output) {

View File

@ -0,0 +1,5 @@
package crosby.binary.file;
public enum CompressFlags {
NONE, DEFLATE
}

View File

@ -1,44 +0,0 @@
# these default settings can be overridden by setting environment variables
PREFIX ?= /usr/local
LIBDIR ?= $(PREFIX)/lib
CXX ?= g++
CXXFLAGS ?= -O3
AR ?= ar
PROTOC ?= protoc
SONAME ?= libosmpbf.so.1
VERSION ?= 1.5.0
CXXFLAGS += -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wno-long-long
.PHONY: clean install
all: libosmpbf.a libosmpbf.so ../include/osmpbf/fileformat.pb.h ../include/osmpbf/osmformat.pb.h
libosmpbf.a: fileformat.pb.o osmformat.pb.o
$(AR) -cr $@ fileformat.pb.o osmformat.pb.o
libosmpbf.so: fileformat.pb.o osmformat.pb.o
$(CXX) $(CPPFLAGS) $(LDFLAGS) -shared -Wl,-soname,$(SONAME) -o $@ fileformat.pb.o osmformat.pb.o
%.pb.o: %.pb.cc
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -fPIC -c -o $@ $<
%.pb.cc ../include/osmpbf/%.pb.h: %.proto
$(PROTOC) --proto_path=. --cpp_out=. $<
cp -v $(subst .proto,.pb.h,$<) ../include/osmpbf/
install:
install -m 755 -d $(DESTDIR)$(LIBDIR)
install -m 644 libosmpbf.a $(DESTDIR)$(LIBDIR)
install -m 755 libosmpbf.so $(DESTDIR)$(LIBDIR)/libosmpbf.so.$(VERSION)
ln -s libosmpbf.so.$(VERSION) $(DESTDIR)$(LIBDIR)/$(SONAME)
ln -s libosmpbf.so.$(VERSION) $(DESTDIR)$(LIBDIR)/libosmpbf.so
install -m 755 -d $(DESTDIR)$(PREFIX)/include/osmpbf
install -m 644 ../include/osmpbf/osmpbf.h $(DESTDIR)$(PREFIX)/include/osmpbf
install -m 644 ../include/osmpbf/fileformat.pb.h $(DESTDIR)$(PREFIX)/include/osmpbf
install -m 644 ../include/osmpbf/osmformat.pb.h $(DESTDIR)$(PREFIX)/include/osmpbf
clean:
rm -f *.pb.h *.pb.cc *.pb.o libosmpbf.a

View File

@ -1,23 +0,0 @@
# these default settings can be overridden by setting environment variables
PREFIX ?= /usr/local
CXX ?= g++
CXXFLAGS ?= -g
CXXFLAGS += -I../include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wno-long-long
LDFLAGS += -L../src -pthread -lz -lprotobuf -losmpbf
.PHONY: clean install
all: osmpbf-outline
osmpbf-outline: osmpbf-outline.cpp
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -o $@ $< $(LDFLAGS)
install:
install -m 755 -d $(DESTDIR)$(PREFIX)/bin
install -m 755 -s osmpbf-outline $(DESTDIR)$(PREFIX)/bin
clean:
rm -f osmpbf-outline