Tuesday, August 13, 2024

Spring Boot

What is circular/cyclic dependency in spring boot?
  • When two services are interdependent on each other, that is to start one service, we require another service and to start second service, we require first service then we have a circular dependency.
  • Circular dependency is also called as cyclic dependency.
  • We face this dependency generally in constructor injection.
  • We can use @Lazy Annotation in any service constructor to resolve this dependency.
How can we create a prototype Bean in Spring boot?
  • @Component
  • @Scope(“prototype”)
  • Public Class SomeClass{
  • }
How can we ensure quality and maintainability of code In our spring boot project?
  • Ensure code reviews happen within peers.
  • Unit test cases, which provide hundred percent coverage for services at least.
  • Integration test cases.
  • Use appropriate data structures and Algorithms.
  • Follow coding standards.
  • Use design pattern, such as MVC, MVVM or MVP.
  • Test your code thoroughly.
  • Use version control system.
  • Write, clear and concise and Modular code.
  • Follow some documentation for your code.
Spring Boot Application perform slower in the production environment, then the development set up. What steps will you take to address this issue?
  • Auto-wiring
    • Don’t use it for classes with static methods.
    • Avoid auto-wiring collection beans.
      • Private List<MyService> services
      • Manually manage the collection and only auto-wire the class.
    • Use constructor injection instead of field injection.
      • Constructor injection is a recommended For better testability and to avoid issues with final fields.
    • When multiple beans of same type exist, Use @Primary To specify, which bean should be preferred.
      • When you want to explicitly specify, which one among the beams, you want to use, Use @Qualifier.
    • Lazy initialisation
      • Use @Lazy if a bean should only be created when it is first needed. This can improve start-up time and resource usage.
  • Connection Pooling
    • Connection pooling Improves performance by reusing existing database connections Instead of opening a new connection for every request.
    • This optimisation technique involves maintaining a pool of open connections rather than opening a new database connection for each API call.
    • Creating a new connection, each time involve a lot of handshake protocols and set up Which can slow down the API.
    • The reuse of connections Can greatly improve throughput.
    • This reduces The overhead of establishing connections and provides efficient management of database resources.
    • Hickari Connection pool is default choice in spring boot for its high-performance.
    • Ensure that transactions are managed appropriately, specially in spring application, where transactions are typically handled declaratively.(@Transactional)
    • Monitor and analyse connection problems, such as active connections, ideal connections, connection wait time, and pool utilisation.
    • Use try with resources to ensure that Connections are properly closed after use, even in the event of exceptions.
    • Instead of opening a fresh connection, each time clients will use pre-opened connection from a pool and once done with transaction, they can release the connection back to the pool.
    • For server-less architecture Connection management can be a bit more challenging
      • This is because each server-less function Instance, typically opens its own database connection.
      • And because server-less Can scale rapidly, this could potentially lead to a large number of connections that could overwhelm the database.
      • Solutions like AWS RDS proxy And Azure SQL database server less are designed to handle this situation and manage connection pooling for you.
  • Async Processing
    •  Asynchronous processing Allows tasks to run in the background, improving the responsiveness and performance of your application.
    • This is particularly useful for long running tasks that do not need to block the main thread.
    • Asynchronous processing is benefit for Tasks that
      • Are I/O intensive (eg file operations, network calls)
      • Take significant amount of time to complete.
      • Do not require immediate results to proceed.
      • Example
        • An application needs to send confirmation emails to users
        • An application allows users to upload files, And the files need to be processed after uploading.
      • Annotate the main class or a configuration with @EnableAsync
      • Annotate the method of a class which has business logic with @Async and let it return a future or Completable future Object.
      • Manage a thread pool using An executor framework.
  • Asynchronous Logging
    • The main application, thread Can place the log entry into memory buffer, While a separate logging thread Write the log entries To the file or send them to the logging, service.
    • With asynchronous, logging We might loose Some logs, if our application crashes before the logs have been written.
    • We can have a separate process to perform logging instead of the logging being done by the process serving the request.
    • We can use a separate process, service, messaging broker, or a separate thread All together to manage our logging.
    • Use different logging levels(Error,Warn,Info,Debug,Trace) Based on the importance And frequency of the log messages.
    • In Development and testing Use Trace and Debug Levels to gain detailed insights Into the application behaviour and identify bugs.
    • In production, we use Info,Warn, And error levels To monitor the applications, health, and performance without Generating excessive log data.
    • For critical Alerts use fatal for logging critical issues that require immediate attention and may cause the application to terminate.
    • Use Minimal logger instances. Avoid creating too many logger instances. Reuse logger instances whenever possible.
      • Private static final logger = LoggerFactory.getLogger(My.class)
    • Use Mapped Diagnostic Context. Add context information like user IDs, transaction ID to log messages to make the debugging easier without cluttering logs.
      • Use MDC to inject identifier(eg Correlation Id’s) into logs to trace individual request across multiple services.
      • It works like a hash-map It uses key value pairs.
      • Mapped Diagnostic Context(MDC) Is a feature provided by logging frameworks like logback, And log4j That allows you to en which log into is with context, specific information.
      • This context can include details such as user IDs, transaction IDs or session IDs Which are crucial for understanding the flow of a request across different parts of an application.
      • MDC uses thread, local storage, meaning that the context data is Local to the thread That sets it. This ensures that the Contextual information is not shared between threads.
      • The %{key} pattern in logging configuration file will include the MDC value associated with the key in each log message.
      • Always clear MDC values after the request is processed to avoid unintentionally leakage of context, data between requests.
      • Use asynchronous logging To offload logging operations to a separate thread, Reducing latency in the main application thread.
      • Use structural logging to capture logs in a format that is easy to parse and analyse Example, Jason format.
      • Add context, such as request, IDs, user IDs, and session ID to logs for better traceability and debugging.
        • Logging.pattern.console=%d{YYYY-MM-DD HH:mm:ss}=%msg%n
        • Logging.pattern.file=%d{YYYY-MM-DD HH:mm:ss}= %msg%n
    • Exclude sensitive data from logs to ensure security and compliance with data protection regulations.
    • Configure log rotation and archiving to prevent log files from growing indefinitely and Consuming disk space.
    • Utilise tools like ELK stack(Elastic search, logstash, kibana), Splunk or Gary Log First centralised log management and analysis.
    • Temporarily disable Logging in performance – critical code sections, if necessary.
  • Database optimisation.
    • Optimise SQL queries to fetch only necessary data. Use joins and subqueries Wisely to minimise the number of separate database calls.
    • Use inner joint for mandatory relationships
    • Use left joint for optional relationships
    • Ensure proper indexing on join columns.
    • Avoid fetching large text fields Or binary data unless necessary.
    • Use pagination to limit the number of rows returned.
    • Use batch possessing to handle multiple operations in a single transaction.
    • Batch insert, update, or delete operations to minimise the number of database round trips.
    • Use native sql queries For complex operations that are not supported by JPQL.
  • N+1 problem
    • The N plus one problem is a common performance issue that occurs when an application needs to fetch data that is related across multiple tables. It typically happens in ORM(Object relational mapping) Framework like hibernate when a developer unintentionally triggers additional database queries.
    • An N+1 problem occurs if we first make a query to fetch initial data, and then we again make multiple queries to fetch related data.
    • If we have initial N data Then we will have one query for the initial data and N queries for related data.
    • To avoid this, it’s more efficient to fetch the data in a single query, Or in some cases, two queries.
      • One to fetch initial data And in second fetch, all the related data of all initial data all together.
      • Then segregate the related data based on initial data based on some business logic.
    • For Initial query Fetch a list of employees, and we are also fetching departments along with that.
      • @Query(“Select e from Employee e Join Fetch e.department”)
    • When using a join against an entity, association, JPA will generate a join between the parent entity and the child entity tables in the generated SQL statement.The SQL Select clause contains only the employee table columns and not the department ones. To fetch the department table columns, we need to use join fetch instead of join.
    • Create index columns for frequently used in Where,Join, and Order By clauses.
    • Monitor and maintain indexes to avoid unnecessary overhead
      • Use composite indexes for columns frequently queried together.
    • Use subquery to filter data within a main query Efficiently.
    • For any join fetch or fetch From multiple tables, use a single query which can be through join tables.
    • Use a single query For fetching information for multiple ID’s , use where in (1,2,3,4).
  • Caching
    • If we have an end point that is most frequently accessed with the same request parameters, we can avoid repeated database hits by caching the response in Redis or Memcache.
    • We store the result of an Expensive computation, so that we can use it again later without need to redo the Computation.
    • Even a brief Period of caching Can make a significant difference in speed.
    • Helps To reduce repeated database, request for similar data.
    • We can use libraries like Redis,Memcached.
    • Cache results of expensive operations like database queries, API calls, or complex calculations, not simple operations that fast to compute.
    • Set appropriate expiration time(TTL-TimeToLive) For cache entries To ensure data consistency and avoid stale data.
    • Ensure that application can handle cache misses gracefully By falling back to fetching data from The original source.
    • Choose Right Cache Provider
      • Select a cache provider Based on your applications needs. For small scale applications or development environments, In memory caches Like Ehcache or Caffine may suffice. For Larger, Distributed applications, Consider using distributed caches like Redis,Hazelcast,or Aerospike.
    • Set Appropriate Cache Expiration.
      • Configure Time to Live(TTL) and idle Timeouts To ensure cached data does not become stale.
      • This Balances performance Benefits with data consistency. Determine times based on nature of your data and how frequently it changes.
    • Monitoring
      • Use monitoring tools to track cache metrics such as hit/miss ratios, Eviction counts, And cache size. Spring boot actuator Can help monitor these metrics. Regularly review these metrics to find tune your cache configuration For optimal performance.
    • Cache only expensive operations
      • Focus on caching Results of operations That are computationally expensive or involves slow input output, such as database queries, or API calls. Avoid caching Lightweight, operations or data that changes frequently To maximise the efficiency of your cache.
    • Implement proper cache Invalidation
      • Ensure your Application properly invalidates or updates cache entries when underlying data changes.Use annotations like @CacheEvict and @CachePut to manage cache In validation effectively, preventing the use of stale data And ensuring consistency.
  • Pagination
    • Read records in a paged fashion That is first, read first Hundred records, then next hundred records, then next hundred and so on. Getting all records at same time is time consuming.
    • In our API response returns a large amount of data, it can slow things down. Instead, wake the response into smaller, more manageable pages using limit and offset parameters.
    • This can speed up data transfer and reduce Load on client side.
  • Use lightweight JSON serialiser
    • When returning Jason responses from your API, the speed of your serialisation process can make a Noticeable difference in response Times.
    • Use a fast serialisation library To minimise the time spent on converting your data into JSON format.
  • Compression
    • Whatever data we are passing As a request and whatever data, we are getting back as a response, we can add compression.
    • Server Will add compression And client will decompress the data To use it.
    • Using compression We can reduce the amount of data transferred over the network.The client then decompresses The data.
    • We have efficient algorithms like Brotli That provide better compression ratio.
    • Many content delivery networks(CDN) Like cloud flare handles compressions for us, thus offloading The task from our server.
  • Consider using environment, variables or secure vaults to manage sensitive information.
  • We can start by checking the logs for errors.
  • Use monitoring tools to Track the performance metrics like memory usage, CPU load and Response time.
  • Compare configuration between development and production to find differences.
  • Database square is are fine. Tuned for performance.
  • Check any external service used in the production affecting performance or not.
What is the difference between spring, spring boot, spring mvc?
  • Spring boot
    • Spring boot solves the configuration problem Of spring MVC 
      • We need to configure view resolver.
      • We need to add web jar
      • We need to Configure dispatcher servlet
      • We need to do hibernate and JPA configuration.
    • Spring boot bundles, all basic frameworks needed to build an application
      • Spring-core, Beans, Context, AOP
      • Web MVC-Spring MVC
      • Jackson-For Json Implementation
      • Validation-Hibernate Validator, Validation API
      • Embedded servlet container-Tomcat
      • Logging-Log back,slf4j
      • Dev tools
      • Actuator
    • Spring boot comes up with a solution called as auto configuration and starters.
    • Spring boot also provides tools to monitor and trace your applications.
      • Spring boot starter, actuator.
    • Spring boot is a composition of spring mvc and spring.
    • Spring boot is used for individual micro services.
    • Simplify development and production deployment.
    • Minimal configuration convention over configuration.
    • Opinionated Default with override options.
    • Micro services, standalone applications, rapid development.
    • Easier to learn and get started quickly.
    • Reduced boilerplate code, followed convention over configuration.
    • Growing community support.
  • Spring MVC
    • SPRING MVC Framework provides decoupled way of developing web applications. With simple concepts like dispatcher servlet, Model and view resolver, It makes it easy to develop the web applications.
    • Spring MVC is used for web application
    • Is based on model view controller framework.
    • Specialised for building web applications
    • Moderate Configuration(XML OR ANNOTATION)
    • Flexible for Web applications
    • Web applications with clear MVC architecture
    • Moderate to learn for developers, familiar with web development
    • Moderate boilerplate code for web components.
    • Moderate, considering configuration and MVC patterns.
    • Established community, specially for web development.
  • Spring
    • The core problem that spring framework solve is dependency injection.
    • Spring is the problem of duplication or plumbing, the code.
      • Spring JDBC
      • Spring MVC
      • SPRING AOP
      • SPRING ORM
      • SPRING JMS
      • SPRING TEST
    • Spring is used for large ERP systems.
    • Spring is a general purpose frame for enterprise Java applications
    • Extensive configuration is required(XML or Java Based)
    • Highly flexible and customisable
    • Used for large scale enterprise applications with complex requirements.
    • Steeper learning curve.
    • More boiler plate code due to extensive configuration.
    • Slowed development speed due to detailed configuration.
    • Large community support and well established community.
How can we integrate a relational database like My Sql With spring boot application?
  • Add Database and Spring data JPA dependencies In the project, build file.
    • Dependencies for string data JPA and Database and Hibernate if required must be added.
  • Use Application properties file to set up the connection details.
    • Add database URL.
    • Add database username.
    • Add database password.
    • Configure hibernate properties, if required.
  • Create a Spring Boot JPA account repository interface in code implementing JPA repository.
    • Removes Boiler plate code from Application
    • Removes DAO Layer from any application.
    • Add @Component above repository.
  • Create table entities using JPA entity.
    • Convert spring boot domains to spring boot entity.
    • @column,@id,@GeneratedValue to generate automatic ids.
  • Use repository In service to perform CRUD operations.
    • Add Support for JPA repository in service.
    • Inject the repository in service And use methods like findOne(id),findAll(),save(),delete() to perform CRUD operations.
  • Spring data JPA Or spring boot JPA  does the integration between the spring application and JPA
    • JPA stands for Java persistence API
    • JPA is a collection of classes to persist data in database
    • JPA perform a bridge between the object model POJO and relational database entity.
    • JPA is a part of spring data
What is Auto configuration in spring boot?
  • We use starter POM’s(Project Object Model)
    • Spring boot starter Web
    • Spring boot starter data JPA
    • SPRING BOOT STARTER TEST
  • Maven will pull all starter jars.
  • All the spring boot auto configuration, JaR’s Present in file spring– boot–auto configure.jar.
  • There is one jar of Auto configuration.
    • This has folder META-INF
      • It has file spring.factories.
    • The file spring.factories defines all the Auto configuration.
    • The classes become active based on some condition I.e Depending on the dependencies used
      • For example, if we use JPA starter Then JPA Auto-configuration will get activated.
    • Over each configuration class, we have animations like
      • @ConditionalOnBean
      • @ConditionalOnClass
      • @ConditionalOnMissingBean
  • These above annotations, define the missing beans.
  • Without this, Auto configuration
    • We had to configure Dispatcher servlet
    • We had to configure data source
    • Entity Manager factory
    • Transaction factory and so on
  • Auto configuration gives permission to spring boot to configure all the things it finds while scanning the context component, Path.
  • Spring boot configuration Checks for all the spring boot, starter jars based on the dependencies used.
  • For example, if a starter JPA will be enabled only once it has a database or when it wants to add a database.
  • All the configuration of each starter jar package is defined in META-INF/spring.factories
How do we maintain environment specific properties in spring application?
  • We maintain environment specific properties in spring application. In our application. Properties file.
  • Spring cloud config server for centralised properties.
  • Spring.active.profiles property for selecting currently active profiles.
  • We use docker and Kubernetes yml file for Environment specific properties, which are provided by container.
What Parameters can be passed in HTTP Headers?
  • Bearer Token
  • Content Type
  • Accept Header
  • File Info
What is Idempotency?
  • Idempotency is when we are continuously making a request, subsequently, and we are getting a same response.
  • These calls are called as Idempotent calls.
  • Example PUT is an Idempotent call.
How does @RequestMapping work?
  • As soon as our application starts all the end points with @RequestMapping Get registered with dispatcher servlet. The request which comes from client is redirected to controller Or end point by dispatcher servlet.
How does spring boot make decision on which server to use?
  • Spring boot decides based on class Path dependencies
  • If a specific server dependency is present, like tomcat jetty etc Then spring boot configure it As default server. If no server dependency is found, then spring boot default to tomcat Which is included in spring boot starter web.
  • This simplifies our set up and configuration
What is the use of actuator?
  • Actuator is used to configure hystrix dashboard. It is used to check the heartbeat of a system or service. If the heartbeat is not available, it will disconnect the service from the service registry.
  • We can use circuit breaker design pattern to have a fallback mechanism once the service is disconnected.
Which property is used to enable or disable actuator?
  • The property used to enable or disable Actuator, can we defined in spring cloud gateway or application gateway. There we have YAML configuration We can provide hystrix properties To enable spring actuator.
  • Every micro service, we can provide support for spring cloud framework.

Q What is Spring boot? Why did you use spring boot in your project?
  • Spring boot is a spring module.
  • Spring boot is a framework for RAD build.
  • Using spring framework with extra support of Auto configuration and embedded application server like (tomcat, jetty).
  • It provides RAD-rapid application development.
  • It helps us in creating efficient fast standalone applications which you can just run it basically removes a lot of configurations and dependencies.
  • Spring boot is lightweight.
  • Spring boot Provide us with a ready to work Project out of box.
  • Spring boot can easily integrate with other technologies.
  • Spring boot can connect to database very easily.
  • It provides security.
Q What are some of the key components of spring boot?
  • Tomcat
  • Starter dependencies, which are configured automatically.
  • Actuator for health check.
  • CLI used to create Standalone application.
Q How can we achieve RAD using Spring Boot?
  • RAD is a modified waterfall model which focuses on developing software in a short span of time.
  • Phases of RAD are as follows
    • Business modelling
      • Business model is designed for the product to be developed.
    • Data modelling
      • Data model is designed, the relation between these data objects are established using Info gathered in first phase.
    • Process modelling
      • Process model is designed. Process descriptions for adding, deleting, retrieving or Modifying a data object are given.
    • Application generation
      • The actual product is built using coding. Convert process and data models into actual prototypes.
    • Testing and turnover
      • Product is tested and if changes are required then whole process starts again.
Q Is it possible to change the port of embedded tomcat server in spring boot?
  • Put “server.port” property in application.properties.
  • Default port is 8080.
Q Can we override or replace the embedded tomcat server in spring boot?
  • Yes we can replace the embedded tomcat with any other servers by using starter dependencies. Like we can use spring–boot–starter–jetty As a dependency for each project as you need.
  • To exclude a dependency from spring boot starter web dependencies artefact use <exclusion> tag.
    • <exclusions>
    • <exclusion>
    • <groupId>org.springframework.boot</groupId>
    • <artifactId>spring-boot-starter-tomcat</artifactId>
    • </exclusions>
    • </exclusion>
Q Can we disable the default web server in the spring boot application?
  • The major strong point in spring is to provide flexibility to build your application loosely coupled. Spring provides features To disable the web server in a quick configuration. 
  • Yes, we can use the application.properties to configure the web application type i.e spring.main.web–application–type = none.
Q How to disable a specific auto-configuration class?
  • You can use the exclude attribute of @EnableAutoConfiguration, If you Find any specific auto–configuration classes that you do not want are being applied.
  • @EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class})
Q What does the @SpringBootApplication annotation do internally?
  • As per the spring boot doc, the @SpringBootApplication annotation is equivalent to using @Configuration, @EnableAutoConfiguration, and @ComponentScan with their default attributes.
    • Configuration helps us to identify beans for a particular spring dependency injection.
    • EnableAutoConfiguration helps in “AutoScan” if a bean is available in our class Path it will be auto-scanned and added to our set of dependencies.
    • ComponentScan Scans for our components or beans.
  • Spring Boot enables the developer to use a single annotation instead of using multiple. But, as we know, spring provided loosely coupled features that we can use for each individual annotation as per our project needs.
Q How to use a property defined in application.properties file into your Java class?
  • Use the @Value Annotation to access the properties which is defined in the application–properties file.
  • @Value(“${custom.value}”)
  • private String customVal;
Q Explain @RestController annotation in spring boot?
  • @RestController is convenience annotation for creating Restful controllers. It is a specialization of @Component and is auto detected to classpath scanning. It adds the @Controller and @ResponseBody annotations. It converts the response to JSON or XML.
  • Which eliminates the need to annotate every request handling method of the controller class with the @ResponseBody annotation. It is typically used in combination with annotated handler methods based on the @RequestMapping annotation.
  • Indicates that the data returned by each method will be written straight into the response body instead of rendering a template.
Q Difference between @RestController annotation and @Controller in Spring Boot?
  • It is that the response from the web application is generally view (HTML+CSS+JavaScript) because they are intended for human viewers while REST API just returns data in form of JSON or XML because most of the REST clients are programs.
  • Same goes with @RestController and @Controller annotation.
  • @Controller Map of the model object to view or template and makes it human readable.
  • @RestController simply returns the object and object data is directly written into HTTP response as JSON or XML.
Q What is the difference between GetMapping and RequestMapping?
  • Request mapping can be used with GET, POST, PUT, and many other request methods using the method attribute on the annotation. Whereas GetMapping is only an extension of RequestMapping, which helps you to improve clarity on requests.
Q What is the use of profiles in Spring Boot?
  • When developing applications for the enterprise, we typically deal with multiple environments such as Dev, QA, and Prod. The configuration properties for these environments are different.
  • For example, we might be using embedded H2 database for Dev, but Prod could have the proprietary Oracle or DB2. Even if the DBMS Is same across environments, the URLs would definitely be Different.
  • To make this easy and clean, spring has the provision of profiles, to help separate the configuration for each environment. So that instead of maintaining this programmatically, the properties can be kept in separate files such as application-dev.properties and application –prod.properties. The default application.properties points to the currently active profile using spring .profiles.active so that the current configuration is picked up.
What is the use of @value In spring boot.
  • Using this we can read value from properties file
How can we use multiple databases in a service?
  • For different environments, we can use profile and then in each profile configuration. Accordingly, we can activate different databases.
  • For single environment, we can have different data sources and for each data source, we can have database template. We can also use factory pattern to return object of different Data Source.

Validations,Errors & Exceptions

 How can we handle exception in spring boot rest application?

Friday, August 9, 2024

Spring Security

What is spring boot two way SSL?

  • Both server and client Trust each other certificate.
  • Both server and client validate each other Certificate.
  • Server send certificate to client and client also has to send required certificate to server, then only handshake happens.
  • Add certificate to client, JDK key store and verify user.

How are we managing security in our applications?

  • We can use LDAP based or OAuth based authentication.
  • We are using Bearer token with Correlation ID
  • For role based We are using spring security roles based on access lists.
  • For Custom Authentication, we can use annotation based security.
How do we provide role based access control using Spring Security?

Monday, July 29, 2024

Spring MVC

What is the swagger in application?

  • A swagger framework allows developers to create interactive machine and human readable API documentation
  • Swagger is an open source, set of uses, specification and tools for dwelling and describing wasteful API’s documentation

What is the difference between @RestController and @Controller

  • Rest Controller is the Combination of two different annotations.@Controller and @ResponseBody.
  • When we annotate with @RestController We do not need to annotate with @ResponseBody
  • @ResponseBody is required because a REST API Should return the complete HTTP response body.

What are the message converters in spring? What is the default message converter?

  • Spring uses Message converter to transform a type of  data like json to Java objects and vice versa.
  • All message converter, implement HTTPMessageConverter class.
  • Json is the Default converter and string uses Jackson library for this.
  • These converters ensures that request and response are seamlessly transformed, allowing us to work with data naturally.

What are the various media type Annotations in spring? How can we produce or consume or type of a resource from a web service in spring? How can an End point produce a json in Spring?

The various Media type annotations in spring are

  • @Produces
    • It defines the endpoints response type.
    • For example, if end point gives adjacent response, then we can give annotation about the end point as @Produces(“application/json”)
  • @Consumes
    • It defines the MIME media type that is service can consume.
    • If an end point Takes Jason as Request body then @Consumes(“application/json”)

Spring Data

What is JPA?
  • JPA Is also called as Java persistence API
  • It is a standard form oracle to map object to database relations.
  • Provides specification and API for JPA
  • Specification is for the JPA provider and vendors like hibernate, Eclipse link, top link or open JPA.
  • API is for developers.
  • Which create an entity and file persist.xml and DAO.
  • Session factory is replaced with Entity Manager Factory
  • Session becomes Entity Manager.
  • We can perform operation operations like persist, merge, find.
  • It also gives us call back annotations like
    • PerPersist
    • PostPersist
    • PreUpdate
What are the various, advantages and disadvantages of hibernate?
  • Advantages
    • Hibernate is database, independent.
    • Hibernate application developed for one database, for example, My sql can be configured for oracle database. 
    • It provides a layered architecture.
    • Hibernate is an implementation of JPA only.
    • Mapping of domain object to relational databases.
    • Caching Framework.
  • Disadvantages
    • We have to give a separate configuration file.
    • Hard to debug
    • Lots of API to learn.
    • Slower than JDBC.
    • Not suitable against batch processing.
What is association in hibernate?
  • Association is a relationship between two database Tables as entries in our Java classes based on their attributes.
  • It is of four types
    • One to one 
    • One to many
    • Many to one
    • Many to many
  • We use @join column for column reference in main entity.
  • We use @oneToOne(mappedBy=“”) in our referenced entity.
How can we connect more than one databases Or use more than one types of database pools in a spring boot project?
  • Define different data source object beans With respective qualifiers.
  • Define different JDBC template beans with respective qualifiers.
  • Use the implementation respective Datasource along with its Jdbc template to fetch data.
How do we implement Pagination in JPA?
  • Client sends a page size, page number and sorting parameters.
  • We find pageable object and find all method.
    • Pagerequest.of(int page, int page size)
  • We must use JPA repository.
  • In JPA repository Pass this pageable object in findAll method.
What is the difference between JPA repository and CRUD repository?
JpaRepository extends PagingAndSortingRepository that extends CrudRepository.
CrudRepository mainly provides CRUD operations.
PagingAndSortingRepository provide methods to perform pagination and sorting of records.
JpaRepository provides JPA related methods such as flushing the persistence context and deleting of records in batch.
Due to their inheritance nature, JpaRepository will have all the behaviors of CrudRepository and PagingAndSortingRepository. So if you don't need the repository to have the functions provided by JpaRepository and PagingAndSortingRepository , use CrudRepository.

Friday, July 26, 2024

Spring Scheduling Tasks

How can we Schedule Cron Expression in Spring Boot?
  • We can use @Scheduled(cron=“@weekly”) at the top of the method.
  • @EnableScheduleing at to of service.
  • @Schedule(cron=“09** SUN”)
  • Enable scheduling tells the spring that our product has methods which needs to be scheduled.
  • Scheduled annotation takes a  cron expression And executes them As per the cron schedule

Thursday, July 25, 2024

Spring AOP

 What is spring AOP?

  • Spring AOP is aspect oriented programming
  • All layers have some aspects in common like security, profiling, logging, transaction management
  • Focus is on aspects enhancement, their reusability, Cross Cutting concerns
  • Crosscutting concern is a common functionality which is required in entire apps or across the apps.

Monday, July 22, 2024

Situations

  • There is a service, which is providing data with many columns but we require only 10 columns. The service fetches data for many joins. We have the database available with us. We can directly use hibernate calls for getting data which can help in reducing the time complexity.
  • Data Inconsistency is to be handled when one services and other updates, the record
    • We can save the progress data in non-relational database like Mango DB
    • Request, service will broadcast and error to topic in Kafka Which will be consumed by other services to perform specific cleanup.
    • We can also have schedule running that will check for transactions failed, and will retry them and if A transaction is not success, then we retry For specific attempts and then leave it And send response in a log.

Tuesday, July 2, 2024

Spring Boot Docker

How can we scale our application in docker at runtime?
  • we can deploy multiple images of same service.
  • We can use Joule and Dribble to scale microservices.

Spring Message Brokers

 In Kafka, how can we consume and produce messages Between services?

  • Kafka consumer and producer configuration Where we provide the topics which we want to consume from and the topics where we want to produce messages to.
  • In This Kafka configuration producer configuration,We can set the bootstrap URL And key serialiser and value serialiser and Use Kafka template.
  • For listener service We can configure the kafka listener Configuration which consumes messages.

Spring Cloud

What is spring cloud configuration server?

We do not have to keep any configuration file in any of the micro services. All services fetch data from the centralised GitHub repository.

How can we modify an existing spring boot application to convert into a server-less architecture?
  • Break down the spring boot monolithic application into the smaller and self-contained services and containerise using docker.
  • Refactor the code of each of the services To work as independent server less function. Removing unnecessary dependencies.
  • Configure event triggers for these function functions Like defining runtime parameter and deploy them to chosen server less platform.
  • Set up API gateway, if needed.
  • Handle data storage with server less database and ensure proper testing, monitoring and security policies and optimising of cost.
  • Example

Friday, June 28, 2024

Hibernate

What does hibernate uses reflection API for?

  • Hibernate uses reflection extensively under the hood to map Java objects to Database tables And vice versa.

 In hibernate, how can we get data from the database?

  • We can get using session object.
  • The methods are get and load.
  • We have a query and criteria interfaces Based on which we can get the object.
What methods are available to get data from database in hibernate?
  • We have system object via which we can get data from database.
  • Methods are get and load.
  • We have query and criteria interfaces to get an object.
  • Load method with a specifier returns, a proxy object of that entity.
  • When we perform a method, it goes to database and get the actual data.
What is the difference between get and load?
  • When we use the load method with any specifier, it will not give you actual object, but proxy object.
  • When we use get method, it will go to database and Fire native query To get actual data.
What are the three states of hibernate entity?
  • New or transient, attached, rested or detached state.
  • When an object is not attached With any Session, it is just created. It is in New or transient state.
    • Not yet attached with session.
  • If an object is associated with Session, then it is said to be in attached state.
  • Once we perform any operations and commit, it is again not held by any session. It is in detached state.

Service Health and Recovery

 How do we know, service is running or not?

  • Service discovery maintains the information of all services.
  • We implement using Eureka server
  • Maintains the status of every service, like it is up/down
  • Every request first goes to service discovery, then service is hit from API gateway.

What Is the use of actuator?

  • Actuator is used to configure hysteric dashboard. Main role of actuator is To check the heartbeat of a system in micro service environment, whether the system is alive or not.
  • If any service is not responding or sending heartbeat for a particular interval, it will disconnect from service, registry, server, or any other mechanisms.
  • From hystrix dashboard We can check whether any service is live or not.
  • We can use a circuit breaker pattern Which is the design pattern to increase the performance or to have a fallback mechanism.
  • The property used to enable or disable actuator is management.end points.web.exposure.include=*
How can we monitor applications and manage performance?
  • For application monitoring, we can use actuator. For service monitoring, we can use zipkins.
  • From zipkins Dashboard, we can check logs.
  • We can also use cloud watch or ELK to check logs to manage performance.
What is circuit breaker method? 
  • When a micro Service feels we can send a default message or we can define a default fallback method, and we can define hystrix properties. This will break the circuit and it will call default method and default message which we have defined.
  • The circuit breaks Depending upon the properties, for example, after her, a certain number of request or after a certain time out the default fallback method or Message is called.
How can we check the health of a service that is service is down or not?
Using hystrix And circuit breaker, where we can define fallback methods. In fallback method, we have the properties like set the time out milliseconds And number of request. Based on these, it will hit the default Method instead of original one.

Wednesday, June 19, 2024

Microservices

What are the some of disadvantages of micro services?
  • Coordination complexity
    • As the micro service architecture comprises different services, communicating between them, these services become a little more complex.
    • Increased development time
    • Requires more complex architecture
    • Set up new architecture
    • Higher cost of implementation

Can we create micro services as state machines?
  • Micro services are independent entities that solve a specific context.
  • For that context, the micro service can work as a state machine.
  • In a state machine, there are life cycle events that cause change in the state of the system.
    • In a library, service, there is a book that changes state based on different events like procure a book, register a book, issue a book, return book, loser, book, lead to return of book, et cetera.
    • These events and book can form a state machine for library micro service.
How we can use multiple databases in spring micro services?
  • For different environment, we can use profile to use multiple databases.
  • Multiple databases for single environment can be configured using Java configuration.
    • We can have different data sources and object.
    • We can have factory method Which will give us different object based on conditions.
How can we scale micro services?
  • We can use Joule and Dribble to scale micro services.
What are some of the advantages of using micro services?
  • Single chain of responsibility(Scale up) Is better.
  • Easy to code and integrate.
  • Deployment takes less time as only concerned micro service needs to be deployed.
How can we perform session management in micro services?
  • We can use SAGA design pattern To perform session management in micro services.
    • We can apply with choreography approach or orchestration approach.
    • In choreography We can use event sourcing approach.
  • We can also manage session using correlation ID between micro services using mapped diagnostics context.
  • Asynchronous Session Can be managed using a broker like Kafka or Rabbit Mq etc.
What is Load Balancing?
  • Load balancing means distributing your incoming request to multiple server instances.

Saturday, June 8, 2024

Difference between Rabbit MQ and Kafka

Rabbit MQ Kafka
Rabbit MQ is basic Queue based messaging system based on FIFO Kafka adds on to the functionality of Rabbit MQ with Stream Processing System.
We use Advance message queueing protocol(AMQP) Kafka does not use Any such protocol or library.
Rabbit MQ has a smart broker, which allows routing, content based routing, Database routing. Kafka has A low level of broker comparatively. 
Rabbit MQ has a exchange, which can route messages to queues based on key. In Kafka, we publish a message and all the subscribers subscribed to the topic get the message.
Rabbit MQ delivers messages to consumers attached to the queue in round robin Fashion In database, we can set up change data capture(CDC) And send data to Kafka.
Rabbit MQ has a property prefetch To check the status of Queue.
We can define our own connectors to connect to legacy systems in Kafka.
Rabbit MQ has acknowledgement mechanism which does not auto delete message until acknowledgement comes. Kafka is basically Designed on fan out mechanism.
Rabbit MQ is an open source message distributor That works like post office and cloud. We can create a consumer group as all the consumers will get the same message.
Developed in 2007 and written in erlang programming language We can create multiple partitions within a topic. Kaka will mod key with Number of partitions To select partition, Number to Push message too.
Rabbit MQ allows micro services to communicate as-synchronously. The number of consumers must be equal to number of partitions in a consumer group.
Kafka deliver message in round robin fashion to each Partition in a consumer group.
We cannot retain messages in Rabbit MQ. We can retain messages in Kafka
Rabbit MQ is a Message Broker basically. How ever we can enhance it to perform Pub Sub Functionality. Kafka has Consumer offset via which we can check what messages has been delivered and what to deliver next.Rather than relying on a message queue, Kafka appends messages to the log and leaves them there, where they remain until the consumer reads it or reaches its retention limit.
RabbitMQ is an open-source distributed message broker that facilitates efficient message delivery in complex routing scenarios.It’s called “distributed” because RabbitMQ typically runs as a cluster of nodes where the queues are distributed across the nodes — replicated for high availability and fault tolerance. Kafka is purely based on Publish Subscribe Architecture geared towards streams and high-ingress data replay
RabbitMQ employs a push model and prevents overwhelming users via the consumer configured prefetch limit. This model is an ideal approach for low-latency messaging. It also functions well with the RabbitMQ queue-based architecture. Think of RabbitMQ as a post office, which receives, stores, and delivers mail, whereas RabbitMQ accepts, stores, and transmits binary data messages. Kafka employs a “pull-based” approach, letting users request message batches from specific offsets. Users can leverage message batching for higher throughput and effective message delivery.

Sunday, April 21, 2024

Configure ELK Stack on Fedora

  • Information about ELK can be found on https://springimplant.blogspot.com/p/elk-stack.html
  • Installing ELK on Fedora
    • Install Java
    • Add ELK to stack repository 
      • cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo
      • [elasticsearch-8.x]
      • name=Elasticsearch repository for 8.x packages
      • baseurl=https://artifacts.elastic.co/packages/8.x/yum
      • gpgcheck=1
      • gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
      • enabled=1
      • autorefresh=1
      • type=rpm-md
      • EOF
    • Import GPG Key
      • sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
    • Install elasticsearch
      • sudo yum -y install vim elasticsearch
    • Enable Service
      • sudo systemctl enable --now elasticsearch.service
    • Check Elastic
      • sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200
    • Create a test index:
      • sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -X PUT "https://localhost:9200/mytest_index" -u elastic
    • Configure elastic user pasword
      • sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i
    • Store Password in Envirnoment Shell
      • export ELASTIC_PASSWORD="your_password"
    • Generate new Enrollment Token
      • /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
    • On your new Elasticsearch node, pass the enrollment token as a parameter to the elasticsearch-reconfigure-node tool
      • /usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <enrollment-token>
    • Install Kibana
      • sudo yum -y install kibana
    • Start Kibana Status
      • sudo systemctl restart kibana.service
    • Reload systemd manager configuration
      • sudo /bin/systemctl daemon-reload
    • Don't change any settings now make sure elasticsearch is running and disable other servers like nginx etc.
    • Check the kibana url.
      • http://localhost:5601/
    • Generate Enrollment Token for Kibana
      • sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
    • Paste Token in Kibana url http://localhost:5601/ then hit “Configure Elastic”.A verification code is generated. Run the command below to retrieve the code.
      • sudo /usr/share/kibana/bin/kibana-verification-code
    • Type the code from command output.Let the configuration to complete and Authenticate with elastic as username and password configured earlier.
    • Install Logstash
      • sudo yum -y install logstash
    • Install other ELK tools
      • sudo yum install filebeat auditbeat metricbeat packetbeat heartbeat-elastic
  • ELK Configuration File Changes
    • /etc/elasticsearch/elasticsearch.yml
      •  # ======================== Elasticsearch Configuration =========================  
         #  
         # NOTE: Elasticsearch comes with reasonable defaults for most settings.  
         #    Before you set out to tweak and tune the configuration, make sure you  
         #    understand what are you trying to accomplish and the consequences.  
         #  
         # The primary way of configuring a node is via this file. This template lists  
         # the most important settings you may want to configure for a production cluster.  
         #  
         # Please consult the documentation for further information on configuration options:  
         # https://www.elastic.co/guide/en/elasticsearch/reference/index.html  
         #  
         # ---------------------------------- Cluster -----------------------------------  
         #  
         # Use a descriptive name for your cluster:  
         #  
         cluster.name: my-application  
         #  
         # ------------------------------------ Node ------------------------------------  
         #  
         # Use a descriptive name for the node:  
         #  
         node.name: node-1  
         #  
         # Add custom attributes to the node:  
         #  
         #node.attr.rack: r1  
         #  
         # ----------------------------------- Paths ------------------------------------  
         #  
         # Path to directory where to store the data (separate multiple locations by comma):  
         #  
         path.data: /var/lib/elasticsearch  
         #  
         # Path to log files:  
         #  
         path.logs: /var/log/elasticsearch  
         #  
         # ----------------------------------- Memory -----------------------------------  
         #  
         # Lock the memory on startup:  
         #  
         #bootstrap.memory_lock: true  
         #  
         # Make sure that the heap size is set to about half the memory available  
         # on the system and that the owner of the process is allowed to use this  
         # limit.  
         #  
         # Elasticsearch performs poorly when the system is swapping the memory.  
         #  
         # ---------------------------------- Network -----------------------------------  
         #  
         # By default Elasticsearch is only accessible on localhost. Set a different  
         # address here to expose this node on the network:  
         #  
         network.host: 0.0.0.0  
         #  
         # By default Elasticsearch listens for HTTP traffic on the first free port it  
         # finds starting at 9200. Set a specific HTTP port here:  
         #  
         http.port: 9200  
         #  
         # For more information, consult the network module documentation.  
         #  
         # --------------------------------- Discovery ----------------------------------  
         #  
         # Pass an initial list of hosts to perform discovery when this node is started:  
         # The default list of hosts is ["127.0.0.1", "[::1]"]  
         #  
         discovery.seed_hosts: []  
         #  
         # Bootstrap the cluster using an initial set of master-eligible nodes:  
         #  
         #cluster.initial_master_nodes: ["node-1", "node-2"]  
         #  
         # For more information, consult the discovery and cluster formation module documentation.  
         #  
         # ---------------------------------- Various -----------------------------------  
         #  
         # Allow wildcard deletion of indices:  
         #  
         #action.destructive_requires_name: false  
         #----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------  
         #  
         # The following settings, TLS certificates, and keys have been automatically     
         # generated to configure Elasticsearch security features on 11-05-2024 16:26:37  
         #  
         # --------------------------------------------------------------------------------  
         # Enable security features  
         xpack.security.enabled: false  
         xpack.security.enrollment.enabled: true  
         # Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents  
         xpack.security.http.ssl:  
          enabled: true  
          keystore.path: certs/http.p12  
         # Enable encryption and mutual authentication between cluster nodes  
         xpack.security.transport.ssl:  
          enabled: true  
          verification_mode: certificate  
          keystore.path: certs/transport.p12  
          truststore.path: certs/transport.p12  
         # Create a new cluster with the current node only  
         # Additional nodes can still join the cluster later  
         cluster.initial_master_nodes: ["springimplant-HP-Notebook"]  
         # Allow HTTP API connections from anywhere  
         # Connections are encrypted and require user authentication  
         http.host: 0.0.0.0  
         # Allow other nodes to join the cluster from anywhere  
         # Connections are encrypted and mutually authenticated  
         #transport.host: 0.0.0.0  
         #----------------------- END SECURITY AUTO CONFIGURATION -------------------------  
    • /etc/kibana/kibana.yml
      •  # For more configuration options see the configuration guide for Kibana in  
         # https://www.elastic.co/guide/index.html  
         # =================== System: Kibana Server ===================  
         # Kibana is served by a back end server. This setting specifies the port to use.  
         server.port: 5601  
         # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.  
         # The default is 'localhost', which usually means remote machines will not be able to connect.  
         # To allow connections from remote users, set this parameter to a non-loopback address.  
         server.host: 0.0.0.0  
         # Enables you to specify a path to mount Kibana at if you are running behind a proxy.  
         # Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath  
         # from requests it receives, and to prevent a deprecation warning at startup.  
         # This setting cannot end in a slash.  
         #server.basePath: ""  
         # Specifies whether Kibana should rewrite requests that are prefixed with  
         # `server.basePath` or require that they are rewritten by your reverse proxy.  
         # Defaults to `false`.  
         #server.rewriteBasePath: false  
         # Specifies the public URL at which Kibana is available for end users. If  
         # `server.basePath` is configured this URL should end with the same basePath.  
         #server.publicBaseUrl: ""  
         # The maximum payload size in bytes for incoming server requests.  
         #server.maxPayload: 1048576  
         # The Kibana server's name. This is used for display purposes.  
         #server.name: "your-hostname"  
         # =================== System: Kibana Server (Optional) ===================  
         # Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.  
         # These settings enable SSL for outgoing requests from the Kibana server to the browser.  
         #server.ssl.enabled: false  
         #server.ssl.certificate: /path/to/your/server.crt  
         #server.ssl.key: /path/to/your/server.key  
         # =================== System: Elasticsearch ===================  
         # The URLs of the Elasticsearch instances to use for all your queries.  
         elasticsearch.hosts: ["http://localhost:9200"]  
         # If your Elasticsearch is protected with basic authentication, these settings provide  
         # the username and password that the Kibana server uses to perform maintenance on the Kibana  
         # index at startup. Your Kibana users still need to authenticate with Elasticsearch, which  
         # is proxied through the Kibana server.  
         #elasticsearch.username: "kibana_system"  
         #elasticsearch.password: "pass"  
         # Kibana can also authenticate to Elasticsearch via "service account tokens".  
         # Service account tokens are Bearer style tokens that replace the traditional username/password based configuration.  
         # Use this token instead of a username/password.  
         # elasticsearch.serviceAccountToken: "my_token"  
         # Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of  
         # the elasticsearch.requestTimeout setting.  
         #elasticsearch.pingTimeout: 1500  
         # Time in milliseconds to wait for responses from the back end or Elasticsearch. This value  
         # must be a positive integer.  
         #elasticsearch.requestTimeout: 30000  
         # The maximum number of sockets that can be used for communications with elasticsearch.  
         # Defaults to `Infinity`.  
         #elasticsearch.maxSockets: 1024  
         # Specifies whether Kibana should use compression for communications with elasticsearch  
         # Defaults to `false`.  
         #elasticsearch.compression: false  
         # List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side  
         # headers, set this value to [] (an empty list).  
         #elasticsearch.requestHeadersWhitelist: [ authorization ]  
         # Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten  
         # by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.  
         #elasticsearch.customHeaders: {}  
         # Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.  
         #elasticsearch.shardTimeout: 30000  
         # =================== System: Elasticsearch (Optional) ===================  
         # These files are used to verify the identity of Kibana to Elasticsearch and are required when  
         # xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.  
         #elasticsearch.ssl.certificate: /path/to/your/client.crt  
         #elasticsearch.ssl.key: /path/to/your/client.key  
         # Enables you to specify a path to the PEM file for the certificate  
         # authority for your Elasticsearch instance.  
         #elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]  
         # To disregard the validity of SSL certificates, change this setting's value to 'none'.  
         #elasticsearch.ssl.verificationMode: full  
         # =================== System: Logging ===================  
         # Set the value of this setting to off to suppress all logging output, or to debug to log everything. Defaults to 'info'  
         #logging.root.level: debug  
         # Enables you to specify a file where Kibana stores log output.  
         logging:  
          appenders:  
           file:  
            type: file  
            fileName: /var/log/kibana/kibana.log  
            layout:  
             type: json  
          root:  
           appenders:  
            - default  
            - file  
         # policy:  
         #  type: size-limit  
         #  size: 256mb  
         # strategy:  
         #  type: numeric  
         #  max: 10  
         # layout:  
         #  type: json  
         # Logs queries sent to Elasticsearch.  
         #logging.loggers:  
         # - name: elasticsearch.query  
         #  level: debug  
         # Logs http responses.  
         #logging.loggers:  
         # - name: http.server.response  
         #  level: debug  
         # Logs system usage information.  
         #logging.loggers:  
         # - name: metrics.ops  
         #  level: debug  
         # Enables debug logging on the browser (dev console)  
         #logging.browser.root:  
         # level: debug  
         # =================== System: Other ===================  
         # The path where Kibana stores persistent data not saved in Elasticsearch. Defaults to data  
         #path.data: data  
         # Specifies the path where Kibana creates the process ID file.  
         pid.file: /run/kibana/kibana.pid  
         # Set the interval in milliseconds to sample system and process performance  
         # metrics. Minimum is 100ms. Defaults to 5000ms.  
         #ops.interval: 5000  
         # Specifies locale to be used for all localizable strings, dates and number formats.  
         # Supported languages are the following: English (default) "en", Chinese "zh-CN", Japanese "ja-JP", French "fr-FR".  
         #i18n.locale: "en"  
         # =================== Frequently used (Optional)===================  
         # =================== Saved Objects: Migrations ===================  
         # Saved object migrations run at startup. If you run into migration-related issues, you might need to adjust these settings.  
         # The number of documents migrated at a time.  
         # If Kibana can't start up or upgrade due to an Elasticsearch `circuit_breaking_exception`,  
         # use a smaller batchSize value to reduce the memory pressure. Defaults to 1000 objects per batch.  
         #migrations.batchSize: 1000  
         # The maximum payload size for indexing batches of upgraded saved objects.  
         # To avoid migrations failing due to a 413 Request Entity Too Large response from Elasticsearch.  
         # This value should be lower than or equal to your Elasticsearch cluster’s `http.max_content_length`  
         # configuration option. Default: 100mb  
         #migrations.maxBatchSizeBytes: 100mb  
         # The number of times to retry temporary migration failures. Increase the setting  
         # if migrations fail frequently with a message such as `Unable to complete the [...] step after  
         # 15 attempts, terminating`. Defaults to 15  
         #migrations.retryAttempts: 15  
         # =================== Search Autocomplete ===================  
         # Time in milliseconds to wait for autocomplete suggestions from Elasticsearch.  
         # This value must be a whole number greater than zero. Defaults to 1000ms  
         #unifiedSearch.autocomplete.valueSuggestions.timeout: 1000  
         # Maximum number of documents loaded by each shard to generate autocomplete suggestions.  
         # This value must be a whole number greater than zero. Defaults to 100_000  
         #unifiedSearch.autocomplete.valueSuggestions.terminateAfter: 100000  
         # This section was automatically generated during setup.  
         elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW5hL2Vucm9sbC1wcm9jZXNzLXRva2VuLTE3MTU0NDY3MzU0MzA6d0ZXRmhPZ3JRWC00MlM1emgtcWM5dw  
         elasticsearch.ssl.certificateAuthorities: [/var/lib/kibana/ca_1715446736140.crt]  
         xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://192.168.1.16:9200'], ca_trusted_fingerprint: 53ea8ad754c940e9b1e7580e87883a57a73813508acc3eb0139e0af8786380d4}]  
    • /etc/filebeat/filebeat.yml
      •  ###################### Filebeat Configuration Example #########################  
         # This file is an example configuration file highlighting only the most common  
         # options. The filebeat.reference.yml file from the same directory contains all the  
         # supported options with more comments. You can use it as a reference.  
         #  
         # You can find the full configuration reference here:  
         # https://www.elastic.co/guide/en/beats/filebeat/index.html  
         # For more available modules and options, please see the filebeat.reference.yml sample  
         # configuration file.  
         # ============================== Filebeat inputs ===============================  
         filebeat.inputs:  
         # Each - is an input. Most options can be set at the input level, so  
         # you can use different inputs for various configurations.  
         # Below are the input-specific configurations.  
         # filestream is an input for collecting log messages from files.  
         - type: filestream  
          # Unique ID among all inputs, an ID is required.  
          id: my-filestream-id  
          # Change to true to enable this input configuration.  
          enabled: false  
          # Paths that should be crawled and fetched. Glob based paths.  
          paths:  
           - /var/log/*.log  
           #- c:\programdata\elasticsearch\logs\*  
          # Exclude lines. A list of regular expressions to match. It drops the lines that are  
          # matching any regular expression from the list.  
          # Line filtering happens after the parsers pipeline. If you would like to filter lines  
          # before parsers, use include_message parser.  
          #exclude_lines: ['^DBG']  
          # Include lines. A list of regular expressions to match. It exports the lines that are  
          # matching any regular expression from the list.  
          # Line filtering happens after the parsers pipeline. If you would like to filter lines  
          # before parsers, use include_message parser.  
          #include_lines: ['^ERR', '^WARN']  
          # Exclude files. A list of regular expressions to match. Filebeat drops the files that  
          # are matching any regular expression from the list. By default, no files are dropped.  
          #prospector.scanner.exclude_files: ['.gz$']  
          # Optional additional fields. These fields can be freely picked  
          # to add additional information to the crawled log files for filtering  
          #fields:  
          # level: debug  
          # review: 1  
         # ============================== Filebeat modules ==============================  
         filebeat.config.modules:  
          # Glob pattern for configuration loading  
          path: ${path.config}/modules.d/*.yml  
          # Set to true to enable config reloading  
          reload.enabled: false  
          # Period on which files under path should be checked for changes  
          #reload.period: 10s  
         # ======================= Elasticsearch template setting =======================  
         setup.template.settings:  
          index.number_of_shards: 1  
          #index.codec: best_compression  
          #_source.enabled: false  
         # ================================== General ===================================  
         # The name of the shipper that publishes the network data. It can be used to group  
         # all the transactions sent by a single shipper in the web interface.  
         #name:  
         # The tags of the shipper are included in their field with each  
         # transaction published.  
         #tags: ["service-X", "web-tier"]  
         # Optional fields that you can specify to add additional information to the  
         # output.  
         #fields:  
         # env: staging  
         # ================================= Dashboards =================================  
         # These settings control loading the sample dashboards to the Kibana index. Loading  
         # the dashboards is disabled by default and can be enabled either by setting the  
         # options here or by using the `setup` command.  
         #setup.dashboards.enabled: false  
         # The URL from where to download the dashboard archive. By default, this URL  
         # has a value that is computed based on the Beat name and version. For released  
         # versions, this URL points to the dashboard archive on the artifacts.elastic.co  
         # website.  
         #setup.dashboards.url:  
         # =================================== Kibana ===================================  
         # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.  
         # This requires a Kibana endpoint configuration.  
         setup.kibana:  
          # Kibana Host  
          # Scheme and port can be left out and will be set to the default (http and 5601)  
          # In case you specify and additional path, the scheme is required: http://localhost:5601/path  
          # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601  
          #host: "localhost:5601"  
          # Kibana Space ID  
          # ID of the Kibana Space into which the dashboards should be loaded. By default,  
          # the Default Space will be used.  
          #space.id:  
         # =============================== Elastic Cloud ================================  
         # These settings simplify using Filebeat with the Elastic Cloud (https://cloud.elastic.co/).  
         # The cloud.id setting overwrites the `output.elasticsearch.hosts` and  
         # `setup.kibana.host` options.  
         # You can find the `cloud.id` in the Elastic Cloud web UI.  
         #cloud.id:  
         # The cloud.auth setting overwrites the `output.elasticsearch.username` and  
         # `output.elasticsearch.password` settings. The format is `<user>:<pass>`.  
         #cloud.auth:  
         # ================================== Outputs ===================================  
         # Configure what output to use when sending the data collected by the beat.  
         # ---------------------------- Elasticsearch Output ----------------------------  
         # output.elasticsearch:  
          # Array of hosts to connect to.  
         # hosts: ["localhost:9200"]  
          # Performance preset - one of "balanced", "throughput", "scale",  
          # "latency", or "custom".  
          preset: balanced  
          # Protocol - either `http` (default) or `https`.  
          #protocol: "https"  
          # Authentication credentials - either API key or username/password.  
          #api_key: "id:api_key"  
          #username: "elastic"  
          #password: "changeme"  
         # ------------------------------ Logstash Output -------------------------------  
         output.logstash:  
          # The Logstash hosts  
          hosts: ["0.0.0.0:5044"]  
          # Optional SSL. By default is off.  
          # List of root certificates for HTTPS server verifications  
          #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]  
          # Certificate for SSL client authentication  
          #ssl.certificate: "/etc/pki/client/cert.pem"  
          # Client Certificate Key  
          #ssl.key: "/etc/pki/client/cert.key"  
         # ================================= Processors =================================  
         processors:  
          - add_host_metadata:  
            when.not.contains.tags: forwarded  
          - add_cloud_metadata: ~  
          - add_docker_metadata: ~  
          - add_kubernetes_metadata: ~  
         # ================================== Logging ===================================  
         # Sets log level. The default log level is info.  
         # Available log levels are: error, warning, info, debug  
         #logging.level: debug  
         # At debug level, you can selectively enable logging only for some components.  
         # To enable all selectors, use ["*"]. Examples of other selectors are "beat",  
         # "publisher", "service".  
         #logging.selectors: ["*"]  
         # ============================= X-Pack Monitoring ==============================  
         # Filebeat can export internal metrics to a central Elasticsearch monitoring  
         # cluster. This requires xpack monitoring to be enabled in Elasticsearch. The  
         # reporting is disabled by default.  
         # Set to true to enable the monitoring reporter.  
         #monitoring.enabled: false  
         # Sets the UUID of the Elasticsearch cluster under which monitoring data for this  
         # Filebeat instance will appear in the Stack Monitoring UI. If output.elasticsearch  
         # is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch.  
         #monitoring.cluster_uuid:  
         # Uncomment to send the metrics to Elasticsearch. Most settings from the  
         # Elasticsearch outputs are accepted here as well.  
         # Note that the settings should point to your Elasticsearch *monitoring* cluster.  
         # Any setting that is not set is automatically inherited from the Elasticsearch  
         # output configuration, so if you have the Elasticsearch output configured such  
         # that it is pointing to your Elasticsearch monitoring cluster, you can simply  
         # uncomment the following line.  
         #monitoring.elasticsearch:  
         # ============================== Instrumentation ===============================  
         # Instrumentation support for the filebeat.  
         #instrumentation:  
           # Set to true to enable instrumentation of filebeat.  
           #enabled: false  
           # Environment in which filebeat is running on (eg: staging, production, etc.)  
           #environment: ""  
           # APM Server hosts to report instrumentation results to.  
           #hosts:  
           # - http://localhost:8200  
           # API Key for the APM Server(s).  
           # If api_key is set then secret_token will be ignored.  
           #api_key:  
           # Secret token for the APM Server(s).  
           #secret_token:  
         # ================================= Migration ==================================  
         # This allows to enable 6.7 migration aliases  
         #migration.6_to_7.enabled: true  
    • We don't need any such configuration for Logstash but we will be setting up pipeline configurations in Logstash to read data from log files and send to elastic servers.
  • Setting up Logstash Pipelines
    • Pipeline consist of three things
      • Input -> Source of data.
      • Filter -> Data not to be sent.
      • Output ->  Where we want to send data.
    • Download some dummy, Apache logs from web these will be our input or source data.
    • Create file apachelog.conf inside /etc/logstash/conf.d folder.
      • You can copy the logstash-sample.conf in /etc/logstash folder or you can use the below sample
      •  input {
          file {
           path => "/home/springimplant/logstash_logs/apache.log"
           start_position => "beginning"
           sincedb_path => "/dev/null"
          }  
         }
         filter {  
          grok {  
              match => {"message" => "%{COMBINEDAPACHELOG}"}  
             }  
          date {  
              match => ["timestamp","dd/MMM/yyyy:HH:mm:ss Z"]  
              }  
         }  
         output {  
          elasticsearch {  
           hosts => ["localhost:9200"]  
           index => "javaimplant-prd-1"  
           # index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"  
           # user => "elastic"  
           # password => "elastic"  
          }  
         }
        
    • This is our pipeline configuration file to inject data into elastic search as discussed earlier it has 3 sections input,filter and output
    • Execute the pipeline using
      • sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/apachelog.conf
    • Below is another example of logstash pipeline configuration file to read data from csv file.
      • The logstash collects data from a static csv file and analyze them using Kibana.
      •  input {  
          file {  
           path => "/home/gauravmatta/logstash_logs/data.csv"  
           start_position => beginning  
          }  
         }  
         filter {  
          csv {  
              columns => [  
                 "time_ref",  
                 "account",  
                 "code",  
                 "country_code",  
                 "product_type",  
                 "value",  
                 "status"  
              ]  
              separator => ","  
            }  
         }  
         output {  
          elasticsearch {  
           hosts => ["localhost:9200"]  
           action => "index"  
           index => "csv-prd-1"  
          }  
         }  
  • Create a view in Kibana
    • Login to Kibana click on discover, click on create a data view, Select time filter field.
    • Click Save Data View to Kibana.
    • Select the date Range of your logs and you will see the necessary statistics.

Saturday, October 21, 2023

Frameworks in Spring

  •  Technology's / frameworks in Spring
    • Spring core
    • Spring MVC
    • Spring boot
    • Spring data
    • Hibernate

Thursday, October 12, 2023

minimum web version required to use JSTL

 Q What is the minimal web version required to use JSTL?

And : 2.4

For example following tag from web.xml uses web 4.0

 <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd" version="4.0">  
 </web-app>  
d

Thursday, September 28, 2023

Create a bean of type java.util.properties

To create a bean of type java.util.properties in Java xml use the following code.


 <bean id="attendkey" class="java.util.Properties" name="attendenceKeys"/>  
 <bean id="attendkey.load" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">  
   <property name="targetObject"><ref bean="attendkey"/></property>  
   <property name="targetMethod"><value>putAll</value></property>  
   <property name="arguments">  
        <props>  
             <prop key="Present">1</prop>  
             <prop key="Absent" >-1</prop>  
       </props>  
  </property>  
 </bean>  

Example :

https://github.com/gauravmatta/springmvc/blob/master/book%20management%20system/src/main/java/com/springimplant/xml/config.xml

Spring Boot

What is circular/cyclic dependency in spring boot? When two services are interdependent on each other, that is to start one service, we requ...