Skip to content

OGProxy - a replacement for iFramely

Announcements
  • And now, changes made to the back-end Proxy Server to increase performance

    Key Changes Made:

    1. Rate Limiting

    Added express-rate-limit to limit requests from a single IP.

    1. Logging

    Integrated morgan for logging HTTP requests.

    1. Health Check Endpoint

    Added a simple endpoint to check the server’s status.

    1. Data Validation

    Implemented input validation for the URL using Joi.

    1. Environment Variables

    Used dotenv for managing sensitive data like API keys and port configuration.

    1. Error Handling

    Enhanced error logging for debugging purposes.

    1. Asynchronous Error Handling

    Utilize a centralized error-handling middleware to manage errors in one place.

    1. Environment Variable Management

    Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.

    1. Static Response Handling

    Use a middleware for handling static responses or messages instead of duplicating logic.

    1. Compression Middleware

    Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.

    1. Timeout Handling on Requests

    Handle timeouts for the requests made to the target URLs and provide appropriate error responses.

    1. Security Improvements

    Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.

    1. Logging Configuration

    Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.

    1. Graceful Shutdown

    Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.

    1. Monitoring and Metrics

    Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.

    1. Response Schema Validation

    Use libraries like Joi or Ajv to validate responses sent back to the client, ensuring they conform to expected formats.

    Again, this new code is running here in test for a few weeks.

  • wowww

    Very good work my friend!!

  • @phenomlab the best of the best, great work Mark 👏🏻👏🏻👏🏻👏🏻.

  • Seeing as not every site on the planet has relevant CORS headers that permit data scraping, I thought I’d make this a bit more obvious on the response. The link is still rendered, but by using the below if the remote site refuses to respond to request

    image.png

    Not entirely sold on the image yet - likely will change it, but you get the idea 🙂 It’s more along the lines of graceful failure rather than a URL that simply does nothing.

  • @phenomlab I love that image and think it is perfect! LOL

  • @phenomlab said in OGProxy - a replacement for iFramely:

    And now, changes made to the back-end Proxy Server to increase performance

    Key Changes Made:

    1. Rate Limiting

    Added express-rate-limit to limit requests from a single IP.

    1. Logging

    Integrated morgan for logging HTTP requests.

    1. Health Check Endpoint

    Added a simple endpoint to check the server’s status.

    1. Data Validation

    Implemented input validation for the URL using Joi.

    1. Environment Variables

    Used dotenv for managing sensitive data like API keys and port configuration.

    1. Error Handling

    Enhanced error logging for debugging purposes.

    1. Asynchronous Error Handling

    Utilize a centralized error-handling middleware to manage errors in one place.

    1. Environment Variable Management

    Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.

    1. Static Response Handling

    Use a middleware for handling static responses or messages instead of duplicating logic.

    1. Compression Middleware

    Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.

    1. Timeout Handling on Requests

    Handle timeouts for the requests made to the target URLs and provide appropriate error responses.

    1. Security Improvements

    Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.

    1. Logging Configuration

    Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.

    1. Graceful Shutdown

    Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.

    1. Monitoring and Metrics

    Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.

    1. Response Schema Validation

    Use libraries like Joi or Ajv to validate responses sent back to the client, ensuring they conform to expected formats.

    Again, this new code is running here in test for a few weeks.

    code updated on github or not ?

  • @DownPW Not yet. One or two bugs left to resolve, then it’ll be posted.

  • Ok I’ll install the old version in the meantime… Or maybe just wait if it’s not too long

  • Can’t wait 😲

    🙂

  • @DownPW Coming soon…


Related Topics
  • Ch..ch..ch..ch..changes!

    Announcements
    16
    15 Votes
    16 Posts
    183 Views

    @phenomlab of course, to be recognised is fantastic.

    @phenomlab said in Ch..ch..ch..ch..changes!:

    Sadly, no. Web crawlers and scrapers are often JS based and read text only, so styles don’t have any bearing.

    I’ve read mixed things about this, but no that does make sense, it was something I read a many years back when using Wordpress.

  • Planned sunset of NTFY plugin

    Pinned Announcements
    7
    8 Votes
    7 Posts
    150 Views

    I’ve noticed that I’m the only one subscribed to the push notifications on this site. If you were using NTFY previously, and have noticed that you’ve not had any alerts for a while, it’s because this feature has been disabled.

    You’ll now need to use the push notification to replace NTFY as mentioned in the first post.

  • ANNOUCEMENT: New NTFY Server

    Announcements
    9
    7 Votes
    9 Posts
    691 Views

    @crazycells that’s as good a test as any 🙂

  • IMPORTANT: Theme / Swatch changes

    Announcements
    4
    6 Votes
    4 Posts
    276 Views

    @cagatay these changes aren’t published anywhere presently, so nothing for you to do.

  • Theme retirement

    Announcements
    21
    16 Votes
    21 Posts
    1k Views

    I relented somewhat here and added another swatch - one I missed, which was previous called “blackout”. This specific one has been adapted to work on the new theming engine, but the others have been reclassified, and renamed to suit.

    image.png

    As a result, the theme you might have had will probably be reflecting something else, so you (might) need to change your themes accordingly. The changes are as follows

    Light -> No Change
    Cloudy -> Is now the old “Dim”
    Dim -> Is now the old “Dark”
    Dark -> Now a new theme based on the revamped “Blackout”

  • Link Not Working

    Solved Customisation
    5
    1 Votes
    5 Posts
    263 Views

    @cagatay Good question, but one that’s likely best answered by the devs themselves. Could easily be done with a simple jQuery regex but that would really just be painting over rotten wood.

  • Testing out Webdock.io

    Moved Announcements
    2
    5 Votes
    2 Posts
    520 Views

    Just coming back to this thread for review (as I often do), and it looks like Webdock have increased their available offerings - some are extremely powerful, yet very competitive from the pricing perspective.

    image.png

    10 CPU cores, plus 20Gb RAM? Well worth a look (and the asking price) - there’s also a fixed IP which is hugely beneficial.

    Clearly, this is well beyond what most people will want to spend - it’s more of an example (but interestingly, Sudonix runs on something not too different from the above).

    However, not all that glitters is gold 😕 - just have a walk through the benchmark report I found below and you’ll see a huge difference between Heztner and Webdock

    https://www.vpsbenchmarks.com/compare/hetzner_vs_webdock

    That being said, the amount of HTTP requests that Webdock handles in relation to Hetzner is superior - @DownPW you might want to have a look at this - there’s a free 24 hour trial… 🙂

    5203639b-2f62-47e6-b87b-37580ce5deae-image.png

  • Fancybox now used for image handling

    Announcements
    16
    6 Votes
    16 Posts
    939 Views

    And it seems to be less conflicting!