[Bug]: Huge performance drop when "allowJs": true is enabled in tsconfig

### Version



at least since 0.3.3, tried in 0.5.3 and looks even worse (no proofs though)



### Steps to reproduce



Minimal reproduction repository : https://github.com/LoganTann/tsjest-performance-issue-reproduction



- Clone the repo

- `npm i`

- `npm run test`





### Expected behavior



- Jest should take 8 seconds to load the dependency, not more than 10 minutes.



### Actual behavior



- Simple test is taking an extremely long time (> 10min) to load the dependency

- Disabling the {allowJs} property in the tsconfig file will make the test run under 8 seconds. However, jest will print over 1400 warnings (executed `grep " file to compile while" output_nojs.log  | wc -l`).



- After a successful run, later tests are expected to be much faster thanks to the caching.



### Debug log



Interrupted execution, took too much time.



### Additional context



- Running in WSL 2, inside the ~/reproduction/ folder

- No performance issues in the back-end, both in dev and prod. The code compiles and runs fine. 

- SWC + vitest is not affected by this issue, but I cannot migrate the whole app in another tech stack.



<details><summary>Performance measures I did five months ago (under 0.3.3)</summary>

<p>





```

cross-env NODE_ENV=test jest -runInBand --maxWorkers=1 SalesforceConnections.controller.spec.ts --no-cache



Imports : 

    HttpService: 62226 ms (@nestjs/axios)

    BadRequestException: 1 ms

    ConfigModule: 4736 ms (@nestjs/config)

    Test: 22756 ms (@nestjs/testing)

    TypeOrmModule: 133894 ms (@nestjs/typeorm)

    Repository: 0 ms

    configuration: 9 ms

    RequestUserModel: 8 ms

    CryptoService: 33 ms

    entities: 153 ms

    FixtureService: 109 ms

    TenantEntity: 0 ms

    TenantModel: 5 ms

    TenantService: 34 ms

    UserEntity: 0 ms

    SalesforceConnectionsController: 183104 ms (imports the same)

    SalesforceInstanceDetailsResponseDto: 0 ms

    SalesforceConnectionEntity: 0 ms

    SalesforceInstanceEntity: 0 ms

    SalesforceConnectionService: 0 ms

    SalesforceInstanceService: 0 ms

    SalesforceOauthService: 0 ms

    ConnectivityStatus: 1 ms

Test module compilation :

    Compile: 11315 ms

```



</p>

</details> 









### Environment



```shell

System:

    OS: Linux 5.15 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)

    CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz

  Binaries:

    Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node

    npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm

  npmPackages:

    jest: ^29.7.0 => 29.7.0

```

[]