OGProxy - a replacement for iFramely
-
Mark could you please install to my site also?
-
@cagatay Let me put the guide together first, and see if you can get it working by yourself. It’s not a difficult installation, and once you understand how the components intersect and work together, I think you’ll be fine. If the worst comes to it however, I’m always happy to help.
-
Already found 2 bugs, which have been committed to live code
- Relative path is provided in some instances, so a function now exists to return the full path instead so the image is rendered
- OGProxy does not target chat - this has been fixed
-
-
One thing I never really took into account when developing OGProxy was the potential for CLS (Cumulative Layout Shifting). If you’re sitting there thinking “what is he talking about?” then this goes some way to explain it
Not only does this harm the user experience in the sense that the page jumps all over the place when you are trying to read content, but is also harmful when it comes to site performance and SEO, as this is a key measurable when checking page performance and speed.
Based on this, I’ve made several changes to how OGProxy works - some of which I will outline below
- Link Selection and Filtering
The function selects certain links in the document and filters out unnecessary ones, like internal links or specific classes.
It further filters links by checking for domains or file types that should be ignored.- Placeholders for Preventing Content Layout Shift (CLS)
Placeholders are inserted initially with a generic “Loading…” message and a temporary image. This prevents CLS, which happens when the page layout shifts due to asynchronously loaded content. By having a placeholder occupy the same space as the final preview, the layout stays stable while data is fetched.
- AJAX Request to Fetch Link Metadata
The function sends an AJAX request to an OpenGraph proxy service to retrieve metadata (title, description, and image) about each link. It uses the same proxy server and API key to fetch this information as it’s predecessor.
- Dynamic Update of Placeholder with Real Data
Once data is retrieved, the placeholder is replaced with the actual preview. Title, description, and image are updated based on the fetched metadata. If an image is missing or invalid, it defaults to a specified “404” image.
- Error Handling and Debugging
Debug logging and error handling ensure that if data can’t be fetched, the placeholders are either left unchanged or logged for troubleshooting.
This approach provides a smoother user experience by managing both loading time and visual stability, which are critical for preventing CLS in dynamically loaded content.
The new code is active on this site, and there’s not only a huge visual improvement, but also serious performance gains. I’ll give it a few weeks, then formally release the new code.
-
And now, changes made to the back-end Proxy Server to increase performance
Key Changes Made:
- Rate Limiting
Added express-rate-limit to limit requests from a single IP.
- Logging
Integrated morgan for logging HTTP requests.
- Health Check Endpoint
Added a simple endpoint to check the server’s status.
- Data Validation
Implemented input validation for the URL using Joi.
- Environment Variables
Used
dotenv
for managing sensitive data like API keys and port configuration.- Error Handling
Enhanced error logging for debugging purposes.
- Asynchronous Error Handling
Utilize a centralized error-handling middleware to manage errors in one place.
- Environment Variable Management
Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.
- Static Response Handling
Use a middleware for handling static responses or messages instead of duplicating logic.
- Compression Middleware
Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.
- Timeout Handling on Requests
Handle timeouts for the requests made to the target URLs and provide appropriate error responses.
- Security Improvements
Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.
- Logging Configuration
Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.
- Graceful Shutdown
Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.
- Monitoring and Metrics
Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.
- Response Schema Validation
Use libraries like
Joi
orAjv
to validate responses sent back to the client, ensuring they conform to expected formats.Again, this new code is running here in test for a few weeks.
-
wowww
Very good work my friend!!
-
And now, changes made to the back-end Proxy Server to increase performance
Key Changes Made:
- Rate Limiting
Added express-rate-limit to limit requests from a single IP.
- Logging
Integrated morgan for logging HTTP requests.
- Health Check Endpoint
Added a simple endpoint to check the server’s status.
- Data Validation
Implemented input validation for the URL using Joi.
- Environment Variables
Used
dotenv
for managing sensitive data like API keys and port configuration.- Error Handling
Enhanced error logging for debugging purposes.
- Asynchronous Error Handling
Utilize a centralized error-handling middleware to manage errors in one place.
- Environment Variable Management
Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.
- Static Response Handling
Use a middleware for handling static responses or messages instead of duplicating logic.
- Compression Middleware
Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.
- Timeout Handling on Requests
Handle timeouts for the requests made to the target URLs and provide appropriate error responses.
- Security Improvements
Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.
- Logging Configuration
Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.
- Graceful Shutdown
Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.
- Monitoring and Metrics
Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.
- Response Schema Validation
Use libraries like
Joi
orAjv
to validate responses sent back to the client, ensuring they conform to expected formats.Again, this new code is running here in test for a few weeks.
@phenomlab the best of the best, great work Mark
.
-
Seeing as not every site on the planet has relevant CORS headers that permit data scraping, I thought I’d make this a bit more obvious on the response. The link is still rendered, but by using the below if the remote site refuses to respond to request
Not entirely sold on the image yet - likely will change it, but you get the idea
It’s more along the lines of graceful failure rather than a URL that simply does nothing.
-
Seeing as not every site on the planet has relevant CORS headers that permit data scraping, I thought I’d make this a bit more obvious on the response. The link is still rendered, but by using the below if the remote site refuses to respond to request
Not entirely sold on the image yet - likely will change it, but you get the idea
It’s more along the lines of graceful failure rather than a URL that simply does nothing.
@phenomlab I love that image and think it is perfect! LOL
-
And now, changes made to the back-end Proxy Server to increase performance
Key Changes Made:
- Rate Limiting
Added express-rate-limit to limit requests from a single IP.
- Logging
Integrated morgan for logging HTTP requests.
- Health Check Endpoint
Added a simple endpoint to check the server’s status.
- Data Validation
Implemented input validation for the URL using Joi.
- Environment Variables
Used
dotenv
for managing sensitive data like API keys and port configuration.- Error Handling
Enhanced error logging for debugging purposes.
- Asynchronous Error Handling
Utilize a centralized error-handling middleware to manage errors in one place.
- Environment Variable Management
Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.
- Static Response Handling
Use a middleware for handling static responses or messages instead of duplicating logic.
- Compression Middleware
Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.
- Timeout Handling on Requests
Handle timeouts for the requests made to the target URLs and provide appropriate error responses.
- Security Improvements
Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.
- Logging Configuration
Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.
- Graceful Shutdown
Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.
- Monitoring and Metrics
Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.
- Response Schema Validation
Use libraries like
Joi
orAjv
to validate responses sent back to the client, ensuring they conform to expected formats.Again, this new code is running here in test for a few weeks.
@phenomlab said in OGProxy - a replacement for iFramely:
And now, changes made to the back-end Proxy Server to increase performance
Key Changes Made:
- Rate Limiting
Added express-rate-limit to limit requests from a single IP.
- Logging
Integrated morgan for logging HTTP requests.
- Health Check Endpoint
Added a simple endpoint to check the server’s status.
- Data Validation
Implemented input validation for the URL using Joi.
- Environment Variables
Used
dotenv
for managing sensitive data like API keys and port configuration.- Error Handling
Enhanced error logging for debugging purposes.
- Asynchronous Error Handling
Utilize a centralized error-handling middleware to manage errors in one place.
- Environment Variable Management
Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.
- Static Response Handling
Use a middleware for handling static responses or messages instead of duplicating logic.
- Compression Middleware
Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.
- Timeout Handling on Requests
Handle timeouts for the requests made to the target URLs and provide appropriate error responses.
- Security Improvements
Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.
- Logging Configuration
Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.
- Graceful Shutdown
Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.
- Monitoring and Metrics
Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.
- Response Schema Validation
Use libraries like
Joi
orAjv
to validate responses sent back to the client, ensuring they conform to expected formats.Again, this new code is running here in test for a few weeks.
code updated on github or not ?
-
@phenomlab said in OGProxy - a replacement for iFramely:
And now, changes made to the back-end Proxy Server to increase performance
Key Changes Made:
- Rate Limiting
Added express-rate-limit to limit requests from a single IP.
- Logging
Integrated morgan for logging HTTP requests.
- Health Check Endpoint
Added a simple endpoint to check the server’s status.
- Data Validation
Implemented input validation for the URL using Joi.
- Environment Variables
Used
dotenv
for managing sensitive data like API keys and port configuration.- Error Handling
Enhanced error logging for debugging purposes.
- Asynchronous Error Handling
Utilize a centralized error-handling middleware to manage errors in one place.
- Environment Variable Management
Use environment variables for more configuration options, such as cache duration or allowed origins, making it easier to change configurations without altering the code.
- Static Response Handling
Use a middleware for handling static responses or messages instead of duplicating logic.
- Compression Middleware
Add compression middleware to reduce the size of the response bodies, which can improve performance, especially for larger responses.
- Timeout Handling on Requests
Handle timeouts for the requests made to the target URLs and provide appropriate error responses.
- Security Improvements
Implement security best practices, such as Helmet for setting HTTP headers, which can help protect against well-known vulnerabilities.
- Logging Configuration
Improve logging with different levels (e.g., info, error) using a logging library like winston, which provides more control over logging output.
- Graceful Shutdown
Implement graceful shutdown logic to handle server termination more smoothly, especially during deployment.
- Monitoring and Metrics
Integrate monitoring tools like Prometheus or an APM tool for better insights into the application’s performance and resource usage.
- Response Schema Validation
Use libraries like
Joi
orAjv
to validate responses sent back to the client, ensuring they conform to expected formats.Again, this new code is running here in test for a few weeks.
code updated on github or not ?
@DownPW Not yet. One or two bugs left to resolve, then it’ll be posted.
-
Ok I’ll install the old version in the meantime… Or maybe just wait if it’s not too long
-
Can’t wait
-
@DownPW Coming soon…
-
A release date maybe ?
-
@DownPW in the coming weeks.
-
@DownPW sorry for the delay here - real life is getting in the way, but the latest release will be pushed out soon.
-
no problem.
I was waiting for this new version to change my server but I think I’m going to install the old one.Keep the good work bro
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (ether email, or push notification). You'll also be able to save bookmarks, use reactions, and upvote to show your appreciation to other community members.
With your input, this post could be even better 💗
RegisterLog in