Eureka Client doesn't change its status to UP

Hi there. We came across a problem that Eureka Client doesn't change its status to _UP_ after registration in Eureka Server (even though `/health` endpoint indicates that it's _UP_).



**Given:**



Spring Boot based application with Spring Cloud support.



Spring Boot - 1.5.4.RELEASE

Spring Cloud - 1.3.1.RELEASE with eureka-client - 1.6.2



_although the same problem was for spring-boot 1.5.9.RELEASE and spring-cloud 1.3.5.RELEASE_



Application configurations:



```

eureka.instance.initialStatus=DOWN

eureka.client.healthcheck.enabled=true

```





**Scenario:**



We deploy a bunch of instances of the same spring boot application. Once all instances are up and running, they're all registered in Eureka Server with initial _DOWN_ status. One by one they should change their status to _UP_.



The problem is that some of them never change their status. 



Usually, there were 2-5 (this number varies randomly from restart to restart) out of 24 instances that never updated their status. The thing is that `/health` endpoint always showed that instance is _UP_ for all health checks registered, but in Eureka UI status was always _DOWN_.





**Investigation:**



It's really hard to reproduce this kind of bug (we could reproduce it only on one of our environments) and I can't provide a simple project to demonstrate this behavior, so I'll just try to describe the problem itself and my thoughts about it.



`DiscoveryClient` uses `HealthCheckHandler` in its `refreshInstanceInfo` method in order to get status for instance. By default, it happens every 30 seconds, and if status has changed, it will be propagated to the Eureka Server on the next heartbeat.



There are two implementations of `HealthCheckHandler` interface. The first one is provided by _Netflix_ which is `HealthCheckCallbackToHandlerBridge`, the other one is provided by _Spring Cloud_ itself which is `EurekaHealthCheckHandler`.



I was able to debug some of my instances at runtime, and it turned out that instances whose status was _DOWN_ were using `HealthCheckCallbackToHandlerBridge` to determine the status, the other ones, whose status was _UP_, were using `EurekaHealthCheckHandler`. 



It seemed really strange, so I went through the code in _eureka-client_ module and took a look at the implementation of `getStatus(InstanceInfo.InstanceStatus currentStatus)` from `HealthCheckCallbackToHandlerBridge`



That's how it looks like:



```

@Override

public InstanceInfo.InstanceStatus getStatus(InstanceInfo.InstanceStatus currentStatus) {

    if (null == callback || InstanceInfo.InstanceStatus.STARTING == currentStatus

            || InstanceInfo.InstanceStatus.OUT_OF_SERVICE == currentStatus) { // Do not go to healthcheck handler if the status is starting or OOS.

        return currentStatus;

    }



    return callback.isHealthy() ? InstanceInfo.InstanceStatus.UP : InstanceInfo.InstanceStatus.DOWN;

}

```



According to this implementation, it was pretty obvious why those instances never changed their status to _UP_. If the first condition is true (which is my case, since I don't have any callback provided),  then the `currentStatus` (which is _DOWN_) will be always returned.





The next question was how come that different implementations of `HealthCheckHandler` are used for different instances of the same application with the same configuration.



Taking look at the code in` spring-cloud-netflix-eureka-client` it can be seen, that `EurekaAutoServiceRegistration` uses `EurekaRegistration` to register it in `EurekaServiceRegistry`.



```

public class EurekaAutoServiceRegistration implements AutoServiceRegistration, SmartLifecycle, Ordered {



	@Override

	public void start() {



		// ... other code omitted



		this.serviceRegistry.register(this.registration);



		this.context.publishEvent(new InstanceRegisteredEvent<>(this, this.registration.getInstanceConfig()));

		this.running.set(true);

	}

	

}

```



During registration process in `EurekaServiceRegistry` the following check is performed:



```

@Override

public void register(EurekaRegistration reg) {



	// ... other code omitted



	if (reg.getHealthCheckHandler() != null) {

		reg.getEurekaClient().registerHealthCheck(reg.getHealthCheckHandler());

	}

}

```



If this condition returns false, then `DiscoveryClient` will use default health check handler which is `HealthCheckCallbackToHandlerBridge` from `netflix-eureka` module.



Now let's see how `EurekaRegistration` gets created (`EurekaClientAutoConfiguration` is responsible for this)



```

// ... other code



@Autowired(required = false)

private HealthCheckHandler healthCheckHandler;



// ... other code



@Bean

@ConditionalOnBean(AutoServiceRegistrationProperties.class)

@ConditionalOnProperty(value = "spring.cloud.service-registry.auto-registration.enabled", matchIfMissing = true)

public EurekaRegistration eurekaRegistration(EurekaClient eurekaClient, CloudEurekaInstanceConfig instanceConfig, ApplicationInfoManager applicationInfoManager) {

	return EurekaRegistration.builder(instanceConfig)

			.with(applicationInfoManager)

			.with(eurekaClient)

			.with(healthCheckHandler)

			.build();

}

```



We can see that `healthCheckHandler` is marked as not required. Actually, it's clear why, cause if `eureka.client.healthcheck.enabled` is false, then `EurekaHealthCheckHandlerConfiguration` never creates `EurekaHealthCheckHandler` bean and potentially it could be null, but not in my case.



**Conclusion:**



Based on the above, it looks like for some reasons `EurekaRegistration` bean gets created before `EurekaHealthCheckHandler` bean. It gets initialized with `healthCheckHandler` field equal to null. Since it's null, `EurekaHealthCheckHandler` won't be registered with `DiscoveryClient` and hence `HealthCheckCallbackToHandlerBridge` will be used as a default health check handler, and it will always return the current status (which is _DOWN_ in my case) for eureka instance.



The interesting thing is that eventually `EurekaHealthCheckHandler` gets created (I was able to see it from `/beans` actuator endpoint) but too late so it's never used. 



Finally, I managed to make it work using the following code:



```

@Configuration

public class EurekaContext {



    @Bean

    @ConditionalOnProperty({"eureka.client.enabled", "eureka.client.healthcheck.enabled"})

    public EurekaHealthCheckHandlerRegistrationListener eurekaHealthCheckHandlerRegistrationListener(EurekaHealthCheckHandler eurekaHealthCheckHandler,

                                                                                                     EurekaClient eurekaClient) {

        return new EurekaHealthCheckHandlerRegistrationListener(eurekaHealthCheckHandler, eurekaClient);

    }



    @Slf4j

    @AllArgsConstructor

    public static class EurekaHealthCheckHandlerRegistrationListener implements ApplicationListener<InstanceRegisteredEvent> {



        private final EurekaHealthCheckHandler eurekaHealthCheckHandler;

        private final EurekaClient eurekaClient;



        @Override

        public void onApplicationEvent(InstanceRegisteredEvent event) {

            if (!(eurekaClient.getHealthCheckHandler() instanceof EurekaHealthCheckHandler)) {

                eurekaClient.registerHealthCheck(eurekaHealthCheckHandler);

            }

        }

    }

}

```



It worked out for me, but it'd be really nice to see it fixed in the upcoming releases. 



I think the most obvious solution would be to add 



`@AutoConfigureAfter(EurekaHealthCheckHandlerConfiguration.class)` to `EurekaClientAutoConfiguration`. 



In this way, we could guarantee, that if `EurekaHealthCheckHandlerConfiguration` is enabled, then `EurekaHealthCheckHandler` will be created and injected into `EurekaClientAutoConfiguration` before `EurekaRegistration` is created.



If you're ok with this and the problem is really how I described it, then I could create a PR.







BTW, I've noticed that in Spring Boot 2.0.0 the process of `EurekaRegistration` creation was refactored and now `healthCheckHandler` is injected in the right way.



```

@Bean

@ConditionalOnBean(AutoServiceRegistrationProperties.class)

@ConditionalOnProperty(value = "spring.cloud.service-registry.auto-registration.enabled", matchIfMissing = true)

public EurekaRegistration eurekaRegistration(EurekaClient eurekaClient, CloudEurekaInstanceConfig instanceConfig, ApplicationInfoManager applicationInfoManager, ObjectProvider<HealthCheckHandler> healthCheckHandler) {

	return EurekaRegistration.builder(instanceConfig)

			.with(applicationInfoManager)

			.with(eurekaClient)

			.with(healthCheckHandler)

			.build();

}

```



So, in this case, `HealthCheckHandler` is coming from `ObjectProvider`, not from _field injection_ and it should work fine (I didn't test it with 2.0.0 version though)



@dsyer @spencergibb  could you please take a look into this? 



Thanks in advance!

[]