docker内服务访问宿主机服务的实现

更新时间：2021年10月21日 08:29:55 作者：failymao

本文主要介绍了docker内服务访问宿主机服务的实现，文中通过示例代码介绍的非常详细，具有一定的参考价值，感兴趣的小伙伴们可以参考一下

1. 场景

使用windows， wsl2 进行日常开发测试工作。但是wsl2经常会遇到网络问题。比如今天在测试一个项目，核心功能是将postgres 的数据使用开源组件synch 同步到clickhouse 这个工作。

测试所需组件

postgres
kafka
zookeeper
redis
synch容器

最开始测试时，选择的方案是，将上述五个服务使用 docker-compose 进行编排， network_modules使用hosts模式，因为考虑到kafka的监听安全机制，这种网络模式，无需单独指定暴露端口。

docker-compose.yaml 文件如下

version: "3"
 
services:
  postgres:
    image: failymao/postgres:12.7
    container_name: postgres
    restart: unless-stopped
    privileged: true                                                      # 设置docker-compose env 文件
    command: [ "-c", "config_file=/var/lib/postgresql/postgresql.conf", "-c", "hba_file=/var/lib/postgresql/pg_hba.conf" ]
    volumes:
      - ./config/postgresql.conf:/var/lib/postgresql/postgresql.conf
      - ./config/pg_hba.conf:/var/lib/postgresql/pg_hba.conf
    environment:
      POSTGRES_PASSWORD: abc123
      POSTGRES_USER: postgres
      POSTGRES_PORT: 15432
      POSTGRES_HOST: 127.0.0.1
    healthcheck:
      test: sh -c "sleep 5 && PGPASSWORD=abc123 psql -h 127.0.0.1 -U postgres -p 15432 -c '\q';"
      interval: 30s
      timeout: 10s
      retries: 3
    network_mode: "host"
 
  zookeeper:
    image: failymao/zookeeper:1.4.0
    container_name: zookeeper
    restart: always
    network_mode: "host"
 
  kafka:
    image: failymao/kafka:1.4.0
    container_name: kafka
    restart: always
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_ZOOKEEPER_CONNECT: localhost:2181
      KAFKA_LISTENERS: PLAINTEXT://127.0.0.1:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092
      KAFKA_BROKER_ID: 1
      KAFKA_LOG_RETENTION_HOURS: 24
      KAFKA_LOG_DIRS: /data/kafka-data  #数据挂载
    network_mode: "host"
 
  producer:
    depends_on:
      - redis
      - kafka
      - zookeeper
    image: long2ice/synch
    container_name: producer
    command: sh -c "
      sleep 30 &&
      synch --alias pg2ch_test produce"
    volumes:
      - ./synch.yaml:/synch/synch.yaml
    network_mode: "host"
 
  # 一个消费者消费一个数据库
  consumer:
    tty: true
    depends_on:
      - redis
      - kafka
      - zookeeper
    image: long2ice/synch
    container_name: consumer
    command: sh -c
      "sleep 30 &&
      synch --alias pg2ch_test consume --schema pg2ch_test"
    volumes:
      - ./synch.yaml:/synch/synch.yaml
    network_mode: "host"
 
  redis:
    hostname: redis
    container_name: redis
    image: redis:latest
    volumes:
      - redis:/data
    network_mode: "host"
 
volumes:
  redis:
  kafka:
  zookeeper:

测试过程中因为要使用 postgres, wal2json组件,在容器里单独安装组件很麻烦，尝试了几次均已失败而告终，所以后来选择了将 postgres 服务安装在宿主机上，容器里面的synch服务使用宿主机的 ip,port端口。

但是当重新启动服务后，synch服务一直启动不起来，日志显示 postgres 无法连接. synch配置文件如下

core:
  debug: true # when set True, will display sql information.
  insert_num: 20000 # how many num to submit,recommend set 20000 when production
  insert_interval: 60 # how many seconds to submit,recommend set 60 when production
  # enable this will auto create database `synch` in ClickHouse and insert monitor data
  monitoring: true
 
redis:
  host: redis
  port: 6379
  db: 0
  password:
  prefix: synch
  sentinel: false # enable redis sentinel
  sentinel_hosts: # redis sentinel hosts
    - 127.0.0.1:5000
  sentinel_master: master
  queue_max_len: 200000 # stream max len, will delete redundant ones with FIFO
 
source_dbs:
  - db_type: postgres
    alias: pg2ch_test
    broker_type: kafka # current support redis and kafka
    host: 127.0.0.1
    port: 5433
    user: postgres
    password: abc123
    databases:
      - database: pg2ch_test
        auto_create: true
        tables:
          - table: pgbench_accounts
            auto_full_etl: true
            clickhouse_engine: CollapsingMergeTree
            sign_column: sign
            version_column:
            partition_by:
            settings:
 
clickhouse:
  # shard hosts when cluster, will insert by random
  hosts:
    - 127.0.0.1:9000
  user: default
  password: ''
  cluster_name:  # enable cluster mode when not empty, and hosts must be more than one if enable.
  distributed_suffix: _all # distributed tables suffix, available in cluster
 
kafka:
  servers:
    - 127.0.0.1:9092
  topic_prefix: synch

这种情况很奇怪，首先确认 postgres, 启动，且监听端口(此处是5433) 也正常，使用localhost和主机eth0网卡地址均报错。

2. 解决

google 答案，参考 stackoverflow 高赞回答，问题解决，原答案如下

If you are using Docker-for-mac or Docker-for-Windows 18.03+, just connect to your mysql service using the host host.docker.internal (instead of the 127.0.0.1 in your connection string).

If you are using Docker-for-Linux 20.10.0+, you can also use the host host.docker.internal if you started your Docker

container with the --add-host host.docker.internal:host-gateway option.

Otherwise, read below

Use** --network="host" **in your docker run command, then 127.0.0.1 in your docker container will point to your docker host.

更多详情见源贴

host 模式下容器内服务访问宿主机服务

将postgres监听地址修改如下 host.docker.internal 报错解决。查看宿主机 /etc/hosts 文件如下

root@failymao-NC:/mnt/d/pythonProject/pg_2_ch_demo# cat /etc/hosts
# This file was automatically generated by WSL. To stop automatic generation of this file, add the following entry to /etc/wsl.conf:
# [network]
# generateHosts = false
127.0.0.1       localhost
 
10.111.130.24    host.docker.internal

可以看到，宿主机 ip跟域名的映射. 通过访问域名，解析到宿主机ip, 访问宿主机服务。

最终启动 synch 服务配置如下

core:
  debug: true # when set True, will display sql information.
  insert_num: 20000 # how many num to submit,recommend set 20000 when production
  insert_interval: 60 # how many seconds to submit,recommend set 60 when production
  # enable this will auto create database `synch` in ClickHouse and insert monitor data
  monitoring: true
 
redis:
  host: redis
  port: 6379
  db: 0
  password:
  prefix: synch
  sentinel: false # enable redis sentinel
  sentinel_hosts: # redis sentinel hosts
    - 127.0.0.1:5000
  sentinel_master: master
  queue_max_len: 200000 # stream max len, will delete redundant ones with FIFO
 
source_dbs:
  - db_type: postgres
    alias: pg2ch_test
    broker_type: kafka # current support redis and kafka
    host: host.docker.internal
    port: 5433
    user: postgres
    password: abc123
    databases:
      - database: pg2ch_test
        auto_create: true
        tables:
          - table: pgbench_accounts
            auto_full_etl: true
            clickhouse_engine: CollapsingMergeTree
            sign_column: sign
            version_column:
            partition_by:
            settings:
 
clickhouse:
  # shard hosts when cluster, will insert by random
  hosts:
    - 127.0.0.1:9000
  user: default
  password: ''
  cluster_name:  # enable cluster mode when not empty, and hosts must be more than one if enable.
  distributed_suffix: _all # distributed tables suffix, available in cluster
 
kafka:
  servers:
    - 127.0.0.1:9092
  topic_prefix: synch    host: host.docker.internal
core:
  debug: true # when set True, will display sql information.
  insert_num: 20000 # how many num to submit,recommend set 20000 when production
  insert_interval: 60 # how many seconds to submit,recommend set 60 when production
  # enable this will auto create database `synch` in ClickHouse and insert monitor data
  monitoring: true
 
redis:
  host: redis
  port: 6379
  db: 0
  password:
  prefix: synch
  sentinel: false # enable redis sentinel
  sentinel_hosts: # redis sentinel hosts
    - 127.0.0.1:5000
  sentinel_master: master
  queue_max_len: 200000 # stream max len, will delete redundant ones with FIFO
 
source_dbs:
  - db_type: postgres
    alias: pg2ch_test
    broker_type: kafka # current support redis and kafka
    host: 
    port: 5433
    user: postgres
    password: abc123
    databases:
      - database: pg2ch_test
        auto_create: true
        tables:
          - table: pgbench_accounts
            auto_full_etl: true
            clickhouse_engine: CollapsingMergeTree
            sign_column: sign
            version_column:
            partition_by:
            settings:
 
clickhouse:
  # shard hosts when cluster, will insert by random
  hosts:
    - 127.0.0.1:9000
  user: default
  password: ''
  cluster_name:  # enable cluster mode when not empty, and hosts must be more than one if enable.
  distributed_suffix: _all # distributed tables suffix, available in cluster
 
kafka:
  servers:
    - 127.0.0.1:9092
  topic_prefix: synch

3. 总结

以--networks="host" 模式下启动容器时，如果想在容器内访问宿主机上的服务，将ip修改为`host.docker.internal`

4. 参考

https://stackoverflow.com/questions/24319662/from-inside-of-a-docker-container-how-do-i-connect-to-the-localhost-of-the-mach

到此这篇关于docker内服务访问宿主机服务的实现的文章就介绍到这了,更多相关docker访问宿主机内容请搜索脚本之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持脚本之家！

您可能感兴趣的文章:

docker与firewalld冲突问题及解决过程
文章指出firewalld与Docker竞争iptables转发链规则,导致互相覆盖,解决方案一是先重启firewalld再重启Docker；二是针对Docker 20.10.0以上版本,将docker0接口移至docker区域以避免冲突
2025-09-09
docker安装RabbitMQ及安装延迟插件的详细过程
MQ(message queue)字面意思上来说消息队列，是一种跨进程的通信机制，用于上下游传递消息，本文给大家详细介绍docker安装RabbitMQ及安装延迟插件的过程，感兴趣的朋友一起看看吧
2022-06-06
docker部署rustdesk远程控制服务器的实现
RustDesk是一款体验优秀的远程控制软件,本文主要介绍了docker部署rustdesk远程控制服务器的实现,具有一定的参考价值,感兴趣的可以了
2024-05-05
如何完全清理你的Docker数据
Docker 不会对你的系统进行任何配置更改……但是它会占用大量的磁盘空间，那么如何完全清理你的数据，本文就详细的来介绍一下，感兴趣的可以了解一下
2021-07-07
docker-compose集成Jenkins部署,打包,发布方式
在Docker环境中安装配置Jenkins,需挂载JDK、Maven等至容器指定路径,设置国内镜像提升下载速度,并添加插件与凭据完成环境配置
2024-10-10
docker安装mongo容器并挂载外部配置文件及目录方式
文章总结：本文详细介绍了如何使用Docker安装MongoDB,并将外部配置文件和目录挂载到容器中,首先,拉取MongoDB镜像,然后创建配置文件和目录,并将配置文件内容复制到容器中,最后,创建容器并运行MongoDB
2025-11-11
docker-compose启动mysql双机热备互为主从的方法实现
本文主要介绍了docker-compose启动mysql双机热备互为主从的方法实现，文中通过示例代码介绍的非常详细，对大家的学习或者工作具有一定的参考学习价值，需要的朋友们下面随着小编来一起学习学习吧
2022-07-07
Docker多阶段构建镜像multi-stage详解
多阶段构建是Docker 17.05及以上版本的新特性,允许在一个Dockerfile中使用多个阶段,每个阶段可以有一个独立的基础镜像,并且可以通过COPY --from命令在阶段之间传递文件,这样可以有效地减少最终镜像的大小,只保留必要的文件
2025-10-10
一次centos Docker网桥模式无法访问宿主机Redis服务的故障排除经历
这篇文章主要给大家介绍了关于一次centos Docker网桥模式无法访问宿主机Redis服务的故障排除经历，文中通过示例代码介绍的非常详细，对大家学习或者使用Docker具有一定的参考学习价值，需要的朋友们下面来一起学习学习吧
2019-10-10
Docker 集成KingBase的详细过程
这篇文章主要介绍了Docker 集成KingBase的详细过程,本文给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友参考下吧
2025-04-04

docker内服务访问宿主机服务的实现

目录

1. 场景

2. 解决

3. 总结

4. 参考

相关文章

最新评论

大家感兴趣的内容

最近更新的内容

常用在线小工具